@@ -0,0 +1,124 @@
|
|||||||
|
---
|
||||||
|
id: "OPS.MON.001"
|
||||||
|
category: "operations/monitoring"
|
||||||
|
revision-version: 1
|
||||||
|
revision-date: "2024-01-14"
|
||||||
|
estimate-time: "1h"
|
||||||
|
necessity: "mandatory"
|
||||||
|
frameworks:
|
||||||
|
- name: "soc2"
|
||||||
|
sections: ["CC2.1", "CC6.1", "CC6.6", "CC6.8", "CC7.1", "CC7.2", "CC7.3"]
|
||||||
|
---
|
||||||
|
|
||||||
|
# Configure logs and implement real-time monitoring
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
It allows you to detect and respond to potential security incidents or system
|
||||||
|
failures immediately, minimizing downtime and reducing the risk of data
|
||||||
|
breaches. It maintain visibility into system performance and security events.
|
||||||
|
|
||||||
|
## Implementation
|
||||||
|
|
||||||
|
If you are only relying on GCP for everything, you can use the GCP solution:
|
||||||
|
|
||||||
|
- Leverage GCP solution
|
||||||
|
|
||||||
|
1. **Install the Monitoring Agent** on your VMs: SSH into your VM and install
|
||||||
|
the agent.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -sSO https://dl.google.com/cloudagents/add-monitoring-agent-repo.sh
|
||||||
|
sudo bash add-monitoring-agent-repo.sh
|
||||||
|
sudo apt-get update
|
||||||
|
sudo apt-get install stackdriver-agent
|
||||||
|
```
|
||||||
|
|
||||||
|
2. In Google Cloud Console, go to **Monitoring** > **Dashboards** and enable
|
||||||
|
**Google Cloud Operations Suite** (formerly Stackdriver).
|
||||||
|
3. Use **Cloud Monitoring** to set up **dashboards**, track key metrics (CPU,
|
||||||
|
memory, latency), and create **alerting policies** for real-time
|
||||||
|
notifications on critical events.
|
||||||
|
|
||||||
|
Else, we recommend either Grafana or Datadog:
|
||||||
|
|
||||||
|
### Implement Datadog with GCP
|
||||||
|
|
||||||
|
1. Go to the [Datadog website](https://www.datadoghq.com/) and sign up for an
|
||||||
|
account.
|
||||||
|
2. **Install the Datadog Agent** on your GCP VMs:
|
||||||
|
|
||||||
|
- **SSH** into your VM.
|
||||||
|
- Run the installation command for the Datadog agent. For Debian/Ubuntu:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
DD_API_KEY=<YOUR_API_KEY> bash -c "$(curl -L https://s3.amazonaws.com/dd-agent/scripts/install_script.sh)"
|
||||||
|
```
|
||||||
|
|
||||||
|
⇒ Replace `<YOUR_API_KEY>` with your Datadog API key (found in the Datadog
|
||||||
|
dashboard under **Integrations** > **APIs**).
|
||||||
|
|
||||||
|
3. **Enable Google Cloud Logging**:
|
||||||
|
- In the **Google Cloud Console**, navigate to **Logging** > **Logs Router**.
|
||||||
|
- Click **Create Sink** and name it **Datadog Logs**.
|
||||||
|
- Set **destination** to **Pub/Sub** and create a new Pub/Sub topic (e.g.,
|
||||||
|
`datadog-logs`).
|
||||||
|
4. **Connect Pub/Sub to Datadog**:
|
||||||
|
- In **Datadog**, go to **Integrations** > **Google Cloud**.
|
||||||
|
- Enable the **Pub/Sub** integration and follow the steps to link it to your
|
||||||
|
GCP project.
|
||||||
|
- Create a service account with the appropriate permissions
|
||||||
|
(`pubsub.subscriber`) and add it to the Pub/Sub topic created.
|
||||||
|
5. **Send Logs to Datadog**:
|
||||||
|
- Configure the Pub/Sub topic to export logs from Google Cloud to Datadog.
|
||||||
|
- Ensure logs from GCP services like Compute Engine, Cloud Functions, etc.,
|
||||||
|
are routed to this topic.
|
||||||
|
6. **Monitor Logs in Datadog**:
|
||||||
|
- In Datadog, navigate to **Logs** to see real-time logs from your GCP
|
||||||
|
infrastructure.
|
||||||
|
- Set up **alerts** or **dashboards** for monitoring specific log patterns or
|
||||||
|
critical events.
|
||||||
|
|
||||||
|
### Implement Grafana with GCP
|
||||||
|
|
||||||
|
- **Create a Grafana Cloud Account**: Sign up at
|
||||||
|
[Grafana Cloud](https://grafana.com/).
|
||||||
|
- **Install the Grafana Agent** on GCP VMs:
|
||||||
|
- **SSH** into your VM.
|
||||||
|
- Download and install the Grafana Agent:
|
||||||
|
```bash
|
||||||
|
sudo curl -O -L https://raw.githubusercontent.com/grafana/agent/main/install-agent.sh
|
||||||
|
sudo bash install-agent.sh
|
||||||
|
```
|
||||||
|
- Configure it with your **Grafana Cloud credentials**.
|
||||||
|
- **Enable Google Cloud Logging**:
|
||||||
|
- In **Google Cloud Console**, go to **Logging** > **Logs Router**.
|
||||||
|
- Create a **Sink** and select **Pub/Sub** as the destination.
|
||||||
|
- Set up a **Pub/Sub topic** for Grafana logs.
|
||||||
|
- **Configure Pub/Sub for Grafana**:
|
||||||
|
- In **Grafana Cloud**, set up **Loki** (for log aggregation).
|
||||||
|
- Configure **Pub/Sub** to forward logs to **Loki** by creating a service
|
||||||
|
account with `pubsub.subscriber` permissions.
|
||||||
|
- **Monitor Logs in Grafana**:
|
||||||
|
- Go to **Explore** in Grafana to view and search logs.
|
||||||
|
- Set up **dashboards** or **alerts** for important log data.
|
||||||
|
|
||||||
|
Once the chosen solution is installed, you need to implement your first
|
||||||
|
dashboard (if not already the case). At least, we think you should be able to
|
||||||
|
follow:
|
||||||
|
|
||||||
|
- Availability:
|
||||||
|
- Resource usage (CPU, memory, storage)
|
||||||
|
- System downtime event
|
||||||
|
- Optional:
|
||||||
|
- Uptime
|
||||||
|
- Latency
|
||||||
|
- Access:
|
||||||
|
- Failed log attemps, unauthorized access, MFA usage
|
||||||
|
- Integrity:
|
||||||
|
- Error rate (eg. rate of 500s)
|
||||||
|
- Data loss or corruption (inc. transaction failures)
|
||||||
|
|
||||||
|
## Evidence
|
||||||
|
|
||||||
|
- Screenshot of the metrics you are tracking.
|
||||||
37
controls/operations/monitoring/OPS.MON.002_keep_your_logs.md
Normal file
37
controls/operations/monitoring/OPS.MON.002_keep_your_logs.md
Normal file
@@ -0,0 +1,37 @@
|
|||||||
|
---
|
||||||
|
id: "OPS.MON.002"
|
||||||
|
category: "operations/monitoring"
|
||||||
|
revision-version: 1
|
||||||
|
revision-date: "2024-01-14"
|
||||||
|
estimate-time: "15m"
|
||||||
|
necessity: "mandatory"
|
||||||
|
frameworks:
|
||||||
|
- name: "soc2"
|
||||||
|
sections: ["CC7.3"]
|
||||||
|
---
|
||||||
|
|
||||||
|
# Keep your logs
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Logs help you to detect and investigate security incidents and troubleshoot
|
||||||
|
operational issues. They provide an audit trail.
|
||||||
|
|
||||||
|
## Implementation
|
||||||
|
|
||||||
|
You should consider a retention period of at least 90 days, and up to 1 year.
|
||||||
|
|
||||||
|
- **If you are using Grafana ⇒ Grafana Loki**: Configure the retention period in
|
||||||
|
`loki-config.yaml`.
|
||||||
|
- **If you rely on GCP**: In **Cloud Logging**, navigate to **Logs Router** and
|
||||||
|
adjust log sink retention settings.
|
||||||
|
|
||||||
|
If you are using Datadog:
|
||||||
|
|
||||||
|
1. Go to **Logs Management**
|
||||||
|
2. Set the retention period (3 to 15 days depending on your plan).
|
||||||
|
3. As you are limited in time, also set up the configuration in GCP.
|
||||||
|
|
||||||
|
## Evidence
|
||||||
|
|
||||||
|
- Screenshot of your configuration in GPC or Grafana
|
||||||
@@ -0,0 +1,74 @@
|
|||||||
|
---
|
||||||
|
id: "OPS.MON.003"
|
||||||
|
category: "operations/monitoring"
|
||||||
|
revision-version: 1
|
||||||
|
revision-date: "2024-01-14"
|
||||||
|
estimate-time: "15m"
|
||||||
|
necessity: "mandatory"
|
||||||
|
frameworks:
|
||||||
|
- name: "soc2"
|
||||||
|
sections: ["CC6.1", "CC6.6", "CC6.8", "CC7.1", "CC7.2", "CC7.3"]
|
||||||
|
---
|
||||||
|
|
||||||
|
# Be automatically notified of issues
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Setting up alerting allows you to respond quickly and minimize downtime or
|
||||||
|
potential data breaches. It ensures that important events, such as unauthorized
|
||||||
|
access or system failures, are addressed promptly.
|
||||||
|
|
||||||
|
## Implementation
|
||||||
|
|
||||||
|
You should implement alerts for critical events (if you have alerts on
|
||||||
|
everything, its not alerting anymore). You should consider:
|
||||||
|
|
||||||
|
- **Resource Utilization**: Monitor high CPU, memory, or disk usage to prevent
|
||||||
|
downtime or system crashes.
|
||||||
|
- **System Downtime**: Set alerts for system unavailability or critical services
|
||||||
|
going offline.
|
||||||
|
- **Network Traffic Anomalies**: Monitor for unusual spikes or patterns in
|
||||||
|
network traffic that could indicate a potential attack.
|
||||||
|
- Optional:
|
||||||
|
- **Unauthorized Access Attempts:** Track and alert on failed logins, unusual
|
||||||
|
login locations, or excessive login attempts.
|
||||||
|
|
||||||
|
To set up an alert:
|
||||||
|
|
||||||
|
### In Cloud Monitoring for GCP:
|
||||||
|
|
||||||
|
1. Navigate to **Monitoring** > **Alerting**.
|
||||||
|
2. Click **Create Policy**.
|
||||||
|
3. Select **Add Condition**, then choose the metric you want to monitor (e.g.,
|
||||||
|
CPU usage).
|
||||||
|
4. Define the alert conditions (e.g., CPU > 80% for 5 minutes).
|
||||||
|
5. Choose how to be notified (email, SMS, **Slack**).
|
||||||
|
6. Name your alert and click **Save** to activate it.
|
||||||
|
|
||||||
|
### In Grafana
|
||||||
|
|
||||||
|
1. Log into your Grafana instance, go to the **Dashboards** section and select
|
||||||
|
the relevant dashboard.
|
||||||
|
2. In the dashboard, add a **panel** that visualizes the metric you want to
|
||||||
|
monitor.
|
||||||
|
3. In the panel settings, go to the **Alert** tab.
|
||||||
|
4. Click **Create Alert** and set conditions (e.g., CPU > 80% for 5 minutes).
|
||||||
|
5. Define the notification channels (e.g., email, Slack) in the **Contact
|
||||||
|
Points** section.
|
||||||
|
6. Save the panel and dashboard. The alert is now active and will notify you
|
||||||
|
when the conditions are met.
|
||||||
|
|
||||||
|
### In Datadog
|
||||||
|
|
||||||
|
1. Go to your Datadog dashboard.
|
||||||
|
2. On the left-hand menu, click on **Monitors** > **New Monitor**.
|
||||||
|
3. Choose the type of monitor (e.g., metric) and select the metric you want to
|
||||||
|
track.
|
||||||
|
4. Define the conditions (e.g., CPU > 80% for 5 minutes).
|
||||||
|
5. Set up notification methods (e.g., email, Slack).
|
||||||
|
6. Name the alert and click **Create Monitor** to activate it.
|
||||||
|
|
||||||
|
## Evidence
|
||||||
|
|
||||||
|
- Screenshot of alerts received on the slack channel/email created for that OR
|
||||||
|
of the alerts created in your solution
|
||||||
Reference in New Issue
Block a user