rohit/probo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add monitoring controls

Browse Source 
Signed-off-by: gearnode <bryan@frimin.fr>
This commit is contained in:
gearnode
2025-01-14 20:30:20 +01:00
parent 8802e09334
commit cf9451db5d
3 changed files with 235 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

124
controls/operations/monitoring/OPS.MON.001_log_and_implement_real_time_monitoring.md Normal file
View File

@@ -0,0 +1,124 @@
---
id: "OPS.MON.001"
category: "operations/monitoring"
revision-version: 1
revision-date: "2024-01-14"
estimate-time: "1h"
necessity: "mandatory"
frameworks:
- name: "soc2"
sections: ["CC2.1", "CC6.1", "CC6.6", "CC6.8", "CC7.1", "CC7.2", "CC7.3"]
---
# Configure logs and implement real-time monitoring
## Purpose
It allows you to detect and respond to potential security incidents or system
failures immediately, minimizing downtime and reducing the risk of data
breaches. It maintain visibility into system performance and security events.
## Implementation
If you are only relying on GCP for everything, you can use the GCP solution:
- Leverage GCP solution
1. **Install the Monitoring Agent** on your VMs: SSH into your VM and install
the agent.
```bash
curl -sSO https://dl.google.com/cloudagents/add-monitoring-agent-repo.sh
sudo bash add-monitoring-agent-repo.sh
sudo apt-get update
sudo apt-get install stackdriver-agent
```
2. In Google Cloud Console, go to **Monitoring** > **Dashboards** and enable
**Google Cloud Operations Suite** (formerly Stackdriver).
3. Use **Cloud Monitoring** to set up **dashboards**, track key metrics (CPU,
memory, latency), and create **alerting policies** for real-time
notifications on critical events.
Else, we recommend either Grafana or Datadog:
### Implement Datadog with GCP
1. Go to the [Datadog website](https://www.datadoghq.com/) and sign up for an
account.
2. **Install the Datadog Agent** on your GCP VMs:
- **SSH** into your VM.
- Run the installation command for the Datadog agent. For Debian/Ubuntu:
```bash
DD_API_KEY=<YOUR_API_KEY> bash -c "$(curl -L https://s3.amazonaws.com/dd-agent/scripts/install_script.sh)"
```
⇒ Replace `<YOUR_API_KEY>` with your Datadog API key (found in the Datadog
dashboard under **Integrations** > **APIs**).
3. **Enable Google Cloud Logging**:
- In the **Google Cloud Console**, navigate to **Logging** > **Logs Router**.
- Click **Create Sink** and name it **Datadog Logs**.
- Set **destination** to **Pub/Sub** and create a new Pub/Sub topic (e.g.,
`datadog-logs`).
4. **Connect Pub/Sub to Datadog**:
- In **Datadog**, go to **Integrations** > **Google Cloud**.
- Enable the **Pub/Sub** integration and follow the steps to link it to your
GCP project.
- Create a service account with the appropriate permissions
(`pubsub.subscriber`) and add it to the Pub/Sub topic created.
5. **Send Logs to Datadog**:
- Configure the Pub/Sub topic to export logs from Google Cloud to Datadog.
- Ensure logs from GCP services like Compute Engine, Cloud Functions, etc.,
are routed to this topic.
6. **Monitor Logs in Datadog**:
- In Datadog, navigate to **Logs** to see real-time logs from your GCP
infrastructure.
- Set up **alerts** or **dashboards** for monitoring specific log patterns or
critical events.
### Implement Grafana with GCP
- **Create a Grafana Cloud Account**: Sign up at
[Grafana Cloud](https://grafana.com/).
- **Install the Grafana Agent** on GCP VMs:
- **SSH** into your VM.
- Download and install the Grafana Agent:
```bash
sudo curl -O -L https://raw.githubusercontent.com/grafana/agent/main/install-agent.sh
sudo bash install-agent.sh
```
- Configure it with your **Grafana Cloud credentials**.
- **Enable Google Cloud Logging**:
- In **Google Cloud Console**, go to **Logging** > **Logs Router**.
- Create a **Sink** and select **Pub/Sub** as the destination.
- Set up a **Pub/Sub topic** for Grafana logs.
- **Configure Pub/Sub for Grafana**:
- In **Grafana Cloud**, set up **Loki** (for log aggregation).
- Configure **Pub/Sub** to forward logs to **Loki** by creating a service
account with `pubsub.subscriber` permissions.
- **Monitor Logs in Grafana**:
- Go to **Explore** in Grafana to view and search logs.
- Set up **dashboards** or **alerts** for important log data.
Once the chosen solution is installed, you need to implement your first
dashboard (if not already the case). At least, we think you should be able to
follow:
- Availability:
- Resource usage (CPU, memory, storage)
- System downtime event
- Optional:
- Uptime
- Latency
- Access:
- Failed log attemps, unauthorized access, MFA usage
- Integrity:
- Error rate (eg. rate of 500s)
- Data loss or corruption (inc. transaction failures)
## Evidence
- Screenshot of the metrics you are tracking.

37
controls/operations/monitoring/OPS.MON.002_keep_your_logs.md Normal file
View File

@@ -0,0 +1,37 @@
---
id: "OPS.MON.002"
category: "operations/monitoring"
revision-version: 1
revision-date: "2024-01-14"
estimate-time: "15m"
necessity: "mandatory"
frameworks:
- name: "soc2"
sections: ["CC7.3"]
---
# Keep your logs
## Purpose
Logs help you to detect and investigate security incidents and troubleshoot
operational issues. They provide an audit trail.
## Implementation
You should consider a retention period of at least 90 days, and up to 1 year.
- **If you are using Grafana ⇒ Grafana Loki**: Configure the retention period in
`loki-config.yaml`.
- **If you rely on GCP**: In **Cloud Logging**, navigate to **Logs Router** and
adjust log sink retention settings.
If you are using Datadog:
1. Go to **Logs Management**
2. Set the retention period (3 to 15 days depending on your plan).
3. As you are limited in time, also set up the configuration in GCP.
## Evidence
- Screenshot of your configuration in GPC or Grafana

74
controls/operations/monitoring/OPS.MON.003_automatically_notified_of_issues.md Normal file
View File

@@ -0,0 +1,74 @@
---
id: "OPS.MON.003"
category: "operations/monitoring"
revision-version: 1
revision-date: "2024-01-14"
estimate-time: "15m"
necessity: "mandatory"
frameworks:
- name: "soc2"
sections: ["CC6.1", "CC6.6", "CC6.8", "CC7.1", "CC7.2", "CC7.3"]
---
# Be automatically notified of issues
## Purpose
Setting up alerting allows you to respond quickly and minimize downtime or
potential data breaches. It ensures that important events, such as unauthorized
access or system failures, are addressed promptly.
## Implementation
You should implement alerts for critical events (if you have alerts on
everything, its not alerting anymore). You should consider:
- **Resource Utilization**: Monitor high CPU, memory, or disk usage to prevent
downtime or system crashes.
- **System Downtime**: Set alerts for system unavailability or critical services
going offline.
- **Network Traffic Anomalies**: Monitor for unusual spikes or patterns in
network traffic that could indicate a potential attack.
- Optional:
- **Unauthorized Access Attempts:** Track and alert on failed logins, unusual
login locations, or excessive login attempts.
To set up an alert:
### In Cloud Monitoring for GCP:
1. Navigate to **Monitoring** > **Alerting**.
2. Click **Create Policy**.
3. Select **Add Condition**, then choose the metric you want to monitor (e.g.,
CPU usage).
4. Define the alert conditions (e.g., CPU > 80% for 5 minutes).
5. Choose how to be notified (email, SMS, **Slack**).
6. Name your alert and click **Save** to activate it.
### In Grafana
1. Log into your Grafana instance, go to the **Dashboards** section and select
the relevant dashboard.
2. In the dashboard, add a **panel** that visualizes the metric you want to
monitor.
3. In the panel settings, go to the **Alert** tab.
4. Click **Create Alert** and set conditions (e.g., CPU > 80% for 5 minutes).
5. Define the notification channels (e.g., email, Slack) in the **Contact
Points** section.
6. Save the panel and dashboard. The alert is now active and will notify you
when the conditions are met.
### In Datadog
1. Go to your Datadog dashboard.
2. On the left-hand menu, click on **Monitors** > **New Monitor**.
3. Choose the type of monitor (e.g., metric) and select the metric you want to
track.
4. Define the conditions (e.g., CPU > 80% for 5 minutes).
5. Set up notification methods (e.g., email, Slack).
6. Name the alert and click **Create Monitor** to activate it.
## Evidence
- Screenshot of alerts received on the slack channel/email created for that OR
of the alerts created in your solution