rohit/probo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update doc location

Browse Source 
Signed-off-by: Bryan Frimin <bryan@getprobo.com>
This commit is contained in:
Bryan Frimin
2025-11-25 11:20:48 +01:00
parent 62a5074e88
commit 8a007148af
5 changed files with 1 additions and 2298 deletions
Download Patch File Download Diff File Expand all files Collapse all files

722
docs/CONFIGURATION.md
View File

@@ -1,722 +0,0 @@
# Probod Configuration Reference
This document provides a comprehensive reference for configuring the Probo compliance management platform daemon (`probod`).
For installation instructions, please refer to the [Installation Guide](./INSTALLATION.md).
## Configuration File Format
Probod uses YAML format for configuration files. The configuration is structured hierarchically with the root key `probod` containing all service-specific settings.
### Basic Configuration Structure
```yaml
unit:
metrics:
addr: "localhost:8081"
tracing:
addr: "localhost:4317"
max-batch-size: 1000
batch-timeout: 10
export-timeout: 10
max-queue-size: 10000
probod:
base-url: "http://localhost:8080"
encryption-key: "base64-encoded-encryption-key"
chrome-dp-addr: "localhost:9222"
api:
addr: "localhost:8080"
cors:
allowed-origins: ["http://localhost:3000"]
extra-header-fields:
"Custom-Header": "value"
pg:
addr: "localhost:5432"
username: "postgres"
password: "postgres"
database: "probod"
pool-size: 100
ca-cert-bundle: |
-----BEGIN CERTIFICATE-----
...certificate content...
-----END CERTIFICATE-----
auth:
disable-signup: false
invitation-confirmation-token-validity: 3600
cookie:
name: "SSID"
domain: "localhost"
secret: "cookie-signing-secret"
duration: 24
password:
pepper: "password-hashing-pepper"
iterations: 1000000
trust-auth:
cookie-name: "TCT"
cookie-domain: "localhost"
cookie-duration: 24
token-duration: 720
report-url-duration: 15
token-secret: "trust-token-signing-secret"
scope: "trust_center_readonly"
token-type: "trust_center_access"
trust-center:
http-addr: ":80"
https-addr: ":443"
aws:
region: "us-east-1"
bucket: "probod"
access-key-id: "access-key"
secret-access-key: "secret-key"
endpoint: "http://127.0.0.1:9000"
notifications:
mailer:
sender-name: "Probo"
sender-email: "no-reply@notification.getprobo.com"
mailer-interval: 60
smtp:
addr: "localhost:1025"
user: "smtp-username"
password: "smtp-password"
tls-required: false
slack:
sender-interval: 60
openai:
api-key: "openai-api-key"
temperature: 0.1
model-name: "gpt-4o"
custom-domains:
renewal-interval: 3600
provision-interval: 30
cname-target: "custom.getprobo.com"
acme:
directory: "https://localhost:14000/dir"
email: "admin@getprobo.com"
key-type: "EC256"
root-ca: ""
account-key: ""
root-ca: ""
connectors:
- provider: "slack"
protocol: "oauth2"
config:
client-id: "slack-client-id"
client-secret: "slack-client-secret"
redirect-uri: "https://localhost:8080/api/console/v1/connectors/complete"
auth-url: "https://slack.com/oauth/v2/authorize"
token-url: "https://slack.com/api/oauth.v2.access"
scopes:
- "chat:write"
- "channels:join"
- "incoming-webhook"
settings:
signing-secret: "slack-signing-secret"
```
## Telemetry and Observability
Probod includes built-in support for Prometheus metrics and OpenTelemetry tracing. The telemetry configuration is part of the main configuration file structure.
### Telemetry Configuration
The telemetry configuration is specified at the top level of the configuration file under the `unit` section:
```yaml
unit:
metrics:
addr: "localhost:8081"
tracing:
addr: "localhost:8082"
max-batch-size: 1000
batch-timeout: 10
export-timeout: 10
max-queue-size: 10000
probod:
# ... rest of probod configuration
```
### Prometheus Metrics
#### `unit.metrics.addr` (string)
**Default**: Same as API address
Network address where the Prometheus metrics endpoint will be exposed. The metrics are available at `/metrics` on this address.
#### Example Configuration
```yaml
unit:
metrics:
addr: "0.0.0.0:8081" # Expose metrics on port 8081
```
### OpenTelemetry Tracing
#### `unit.tracing.addr` (string)
**Default**: Not configured (tracing disabled)
Network address for the OpenTelemetry trace exporter endpoint.
#### `unit.tracing.max-batch-size` (integer)
**Default**: `512`
Maximum number of spans to batch before exporting.
#### `unit.tracing.batch-timeout` (integer)
**Default**: `5` (seconds)
Maximum time to wait before exporting a batch of spans.
#### `unit.tracing.export-timeout` (integer)
**Default**: `30` (seconds)
Maximum time to wait for span export to complete.
#### `unit.tracing.max-queue-size` (integer)
**Default**: `2048`
Maximum queue size for spans waiting to be exported.
#### Example Configuration
```yaml
unit:
tracing:
addr: "tempo:4317" # OTLP gRPC endpoint
max-batch-size: 1000
batch-timeout: 10
export-timeout: 10
max-queue-size: 10000
```
#### Built-in Instrumentation
Probod automatically instruments:
- Database operations (PostgreSQL queries)
- HTTP requests and responses
- GraphQL operations and resolvers
- S3 storage operations
- PDF generation processes
#### Prometheus Scrape Configuration
Configure Prometheus to scrape metrics from probod:
```yaml
# prometheus.yml
global:
scrape_interval: 15s
scrape_configs:
- job_name: "probod"
scrape_interval: 15s
static_configs:
- targets: ["probod:8081"] # Use metrics addr from config
metrics_path: "/metrics"
```
### Logging
Probod provides automatic structured JSON logging with:
- Request correlation IDs
- Integration with OpenTelemetry trace and span IDs
- Component-specific loggers
- Consistent formatting across all services
## Configuration Sections
### General Settings
#### `base-url` (string)
**Default**: `"http://localhost:8080"`
The base URL where the Probod service will be accessible externally. This should include the scheme (http or https), hostname, and optionally port. This setting affects URL generation for emails, redirects, and API responses. For production deployments, use the full HTTPS URL (e.g., `"https://app.example.com"`).
#### `encryption-key` (string)
**Required**
Base64-encoded encryption key used for encrypting sensitive data at rest. Must be provided for production deployments.
#### `chrome-dp-addr` (string)
**Default**: `"localhost:9222"`
Address of the Chrome DevTools Protocol endpoint used for PDF generation and document processing.
### API Configuration
#### `api.addr` (string)
**Default**: `"localhost:8080"`
The network address and port where the Probod API server will listen for incoming connections.
#### `api.cors.allowed-origins` (array of strings)
**Default**: `[]`
List of origins allowed for Cross-Origin Resource Sharing (CORS) requests. Required for web applications accessing the API from different domains.
#### `api.extra-header-fields` (map of string to string)
**Default**: `{}`
Additional HTTP headers to include in API responses. Useful for custom security headers or integration requirements.
### Database Configuration
#### `pg.addr` (string)
**Default**: `"localhost:5432"`
PostgreSQL database server address and port.
#### `pg.username` (string)
**Default**: `"postgres"`
Database username for authentication.
#### `pg.password` (string)
**Default**: `"postgres"`
Database password for authentication.
#### `pg.database` (string)
**Default**: `"probod"`
Name of the PostgreSQL database to connect to.
#### `pg.pool-size` (integer)
**Default**: `100`
Maximum number of database connections in the connection pool.
#### `pg.ca-cert-bundle` (string)
**Optional**
PEM-encoded CA certificate bundle for TLS database connections. Required when connecting to databases with custom or self-signed certificates.
**Environment Variable Options:**
- `PG_CA_BUNDLE`: Provide the CA bundle content directly as an environment variable (suitable for smaller bundles)
- `PG_CA_BUNDLE_PATH`: Provide a file path to the CA bundle (recommended for large CA bundles to avoid "Argument list too long" errors)
**Example using file path:**
```yaml
# docker-compose.yml or Kubernetes deployment
environment:
PG_CA_BUNDLE_PATH: /etc/ssl/certs/ca-certificates.crt
```
**Note:** When using `PG_CA_BUNDLE_PATH`, the file is read during configuration generation, avoiding environment size limitations. This is the recommended approach when using system CA bundles or large certificate collections.
### Authentication Configuration
#### `auth.disable-signup` (boolean)
**Default**: `false`
When set to `true`, disables new user registration through the web interface.
#### `auth.invitation-confirmation-token-validity` (integer)
**Default**: `3600`
Validity period (in seconds) for email invitation confirmation tokens.
#### `auth.cookie.name` (string)
**Default**: `"SSID"`
Name of the HTTP cookie used for session management.
#### `auth.cookie.domain` (string)
**Default**: `"localhost"`
Domain scope for session cookies.
#### `auth.cookie.secret` (string)
**Default**: Auto-generated
Secret key used for signing session cookies. Should be at least 32 bytes for security.
#### `auth.cookie.duration` (integer)
**Default**: `24`
Session cookie lifetime in hours.
#### `auth.cookie.secure` (boolean)
**Default**: `true`
Controls whether the Secure flag is set on session cookies. When true, cookies are only sent over HTTPS connections.
**Important**: This must be set to `true` for SAML authentication to work properly. SAML requires `SameSite=None` cookies for cross-site POST requests from identity providers, and modern browsers require the `Secure` flag to be set when using `SameSite=None`. Setting this to `false` will cause SAML authentication to fail as session cookies will be rejected by browsers.
#### `auth.password.pepper` (string)
**Default**: Auto-generated
Additional secret value used in password hashing. Should be at least 32 bytes and kept confidential.
#### `auth.password.iterations` (integer)
**Default**: `1000000`
Number of iterations for password hashing algorithm (PBKDF2). Higher values increase security but require more computational resources.
### Trust Center Authentication
#### `trust-auth.cookie-name` (string)
**Default**: `"TCT"`
Name of the HTTP cookie used for trust center access tokens.
#### `trust-auth.cookie-domain` (string)
**Default**: `"localhost"`
Domain scope for trust center cookies.
#### `trust-auth.cookie-duration` (integer)
**Default**: `24`
Trust center cookie lifetime in hours.
#### `trust-auth.token-duration` (integer)
**Default**: `720`
Trust center access token lifetime in hours.
#### `trust-auth.report-url-duration` (integer)
**Default**: `15`
Validity period for generated report URLs in minutes.
#### `trust-auth.token-secret` (string)
**Default**: Auto-generated
Secret key used for signing trust center tokens. Should be at least 32 bytes.
#### `trust-auth.scope` (string)
**Default**: `"trust_center_readonly"`
OAuth2 scope for trust center access.
#### `trust-auth.token-type` (string)
**Default**: `"trust_center_access"`
Type identifier for trust center access tokens.
### Trust Center Configuration
#### `trust-center.http-addr` (string)
**Default**: `":80"`
Network address and port where the trust center HTTP server will listen for ACME HTTP-01 challenges and HTTP to HTTPS redirects.
#### `trust-center.https-addr` (string)
**Default**: `":443"`
Network address and port where the trust center HTTPS server will listen for secure connections. This server handles custom domain routing with automatic TLS certificate management.
### AWS Configuration
#### `aws.region` (string)
**Default**: `"us-east-1"`
AWS region for S3 bucket operations.
#### `aws.bucket` (string)
**Default**: `"probod"`
S3 bucket name for file storage.
#### `aws.access-key-id` (string)
**Required**
AWS access key ID for authentication.
#### `aws.secret-access-key` (string)
**Required**
AWS secret access key for authentication.
#### `aws.endpoint` (string)
**Optional**
Custom S3-compatible endpoint URL. Useful for local development with MinIO or other S3-compatible services.
### Notifications Configuration
The `notifications` section configures how Probod sends notifications through various channels.
#### `notifications.mailer.sender-name` (string)
**Default**: `"Probo"`
Display name for outgoing emails.
#### `notifications.mailer.sender-email` (string)
**Default**: `"no-reply@notification.getprobo.com"`
Email address used as the sender for outgoing emails.
#### `notifications.mailer.mailer-interval` (integer)
**Default**: `60`
Interval in seconds between checking for pending email notifications to send.
#### `notifications.mailer.smtp.addr` (string)
**Default**: `"localhost:1025"`
SMTP server address and port.
#### `notifications.mailer.smtp.user` (string)
**Optional**
Username for SMTP authentication.
#### `notifications.mailer.smtp.password` (string)
**Optional**
Password for SMTP authentication.
#### `notifications.mailer.smtp.tls-required` (boolean)
**Default**: `false`
Whether TLS encryption is required for SMTP connections.
#### `notifications.slack.sender-interval` (integer)
**Default**: `60`
Interval in seconds between checking for pending Slack notifications to send.
### OpenAI Integration
#### `openai.api-key` (string)
**Required for AI features**
API key for OpenAI services integration.
#### `openai.temperature` (float)
**Default**: `0.1`
Temperature parameter for AI model responses (0.0 to 1.0). Lower values produce more deterministic outputs.
#### `openai.model-name` (string)
**Default**: `"gpt-4o"`
OpenAI model identifier to use for AI-powered features.
### Custom Domains Configuration
The `custom-domains` section configures automatic TLS certificate management for custom trust center domains using ACME (Let's Encrypt).
#### `custom-domains.renewal-interval` (integer)
**Default**: `3600`
Interval in seconds between checking for certificates that need renewal.
#### `custom-domains.provision-interval` (integer)
**Default**: `30`
Interval in seconds between checking for domains that need certificate provisioning.
#### `custom-domains.cname-target` (string)
**Default**: `"custom.getprobo.com"`
The CNAME target that custom domains should point to. This is used for domain validation and documentation.
#### `custom-domains.acme.directory` (string)
**Default**: `"https://localhost:14000/dir"`
ACME directory URL. For local development, use Pebble at `"https://localhost:14000/dir"`.
#### `custom-domains.acme.email` (string)
**Default**: `"admin@getprobo.com"`
Contact email address for ACME account registration and certificate expiration notifications.
#### `custom-domains.acme.key-type` (string)
**Default**: `"EC256"`
Type of cryptographic key to use for certificates. Supported values: `"EC256"`, `"EC384"`, `"RSA2048"`, `"RSA4096"`.
#### `custom-domains.acme.root-ca` (string)
**Optional**
PEM-encoded root CA certificate for ACME server validation. Required when using private ACME servers or testing with Pebble.
#### `custom-domains.acme.account-key` (string)
**Optional**
PEM-encoded ACME account private key. If not provided, a new account key will be generated automatically.
### External Connectors
The `connectors` section defines external service integrations.
#### OAuth2 Connector Configuration
```yaml
connectors:
- provider: "slack"
protocol: "oauth2"
config:
client-id: "oauth2-client-id"
client-secret: "oauth2-client-secret"
redirect-uri: "https://your-domain/api/console/v1/connectors/complete"
auth-url: "https://slack.com/oauth/v2/authorize"
token-url: "https://slack.com/api/oauth.v2.access"
scopes:
- "chat:write"
- "channels:join"
- "incoming-webhook"
settings:
signing-secret: "slack-signing-secret"
```
##### `connectors[].provider` (string)
**Required**
Service provider name. Supported values: `"slack"`.
##### `connectors[].protocol` (string)
**Required**
Connector protocol type. Currently supported: `"oauth2"`.
##### `connectors[].config.client-id` (string)
**Required**
OAuth2 client identifier provided by the external service.
##### `connectors[].config.client-secret` (string)
**Required**
OAuth2 client secret provided by the external service.
##### `connectors[].config.redirect-uri` (string)
**Required**
OAuth2 redirect URI. Must match the URI registered with the external service.
##### `connectors[].config.auth-url` (string)
**Required**
OAuth2 authorization endpoint URL.
##### `connectors[].config.token-url` (string)
**Required**
OAuth2 token exchange endpoint URL.
##### `connectors[].config.scopes` (array of strings)
**Optional**
List of OAuth2 scopes to request during authorization.
##### `connectors[].settings` (object)
**Optional**
Additional provider-specific settings. For Slack connectors, this includes:
- `signing-secret` (string): Slack signing secret for webhook verification.
## Troubleshooting
### Common Configuration Issues
1. **Database Connection Failures**: Verify database credentials, network connectivity, and certificate configuration.
2. **Authentication Problems**: Check cookie domain settings and secret key configuration.
3. **External Connector Issues**: Verify OAuth2 client credentials and redirect URI configuration.
4. **File Upload Problems**: Ensure AWS credentials and S3 bucket configuration are correct.
### Logging
Probod provides structured logging that can help diagnose configuration issues. Enable debug logging by setting appropriate log levels in your deployment environment.
## Configuration Validation
Probod validates configuration on startup and will report specific errors for:
- Missing required fields
- Invalid data formats
- Unreachable external services
- Invalid secrets or keys
Review startup logs carefully to identify and resolve configuration issues.

203
docs/DOCKER_ENVIRONMENT_VARIABLES.md
View File

@@ -1,203 +0,0 @@
# Environment Variables Reference
This document provides a comprehensive reference for all environment variables used by the Probo entrypoint script to generate the configuration file.
## Configuration File
| Variable | Description | Default Value | Required |
| ------------- | ------------------------------ | ------------------------ | -------- |
| `CONFIG_FILE` | Path to the configuration file | `/etc/probod/config.yml` | No |
## Observability
### Metrics
| Variable | Description | Default Value | Required |
| -------------- | --------------------------------------- | ---------------- | -------- |
| `METRICS_ADDR` | Address for Prometheus metrics endpoint | `localhost:8081` | No |
### Tracing
| Variable | Description | Default Value | Required |
| ------------------------ | ------------------------------------------------------- | ---------------- | -------- |
| `TRACING_ADDR` | OpenTelemetry collector address for distributed tracing | `localhost:4317` | No |
| `TRACING_MAX_BATCH_SIZE` | Maximum number of spans to batch before export | `512` | No |
| `TRACING_BATCH_TIMEOUT` | Timeout in seconds for batching spans | `5` | No |
| `TRACING_EXPORT_TIMEOUT` | Timeout in seconds for exporting traces | `30` | No |
| `TRACING_MAX_QUEUE_SIZE` | Maximum queue size for spans waiting to be exported | `2048` | No |
## Application Configuration
| Variable | Description | Default Value | Required |
| ----------------------- | ---------------------------------------------------------------- | ----------------------- | -------- |
| `PROBOD_BASE_URL` | Public hostname for the Probo instance (used for URL generation) | `http://localhost:8080` | No |
| `PROBOD_ENCRYPTION_KEY` | Base64-encoded encryption key for sensitive data (32+ bytes) | - | **Yes** |
| `CHROME_DP_ADDR` | Chrome DevTools Protocol address for PDF generation | `localhost:9222` | No |
## API Configuration
| Variable | Description | Default Value | Required |
| -------------------------- | ---------------------------------------------- | ----------------------- | -------- |
| `API_ADDR` | Address and port for the API server to bind to | `:8080` | No |
| `API_CORS_ALLOWED_ORIGINS` | Comma-separated list of allowed CORS origins | `http://localhost:8080` | No |
## PostgreSQL Database
| Variable | Description | Default Value | Required |
| -------------- | -------------------------------------------------- | ---------------- | -------- |
| `PG_ADDR` | PostgreSQL server address and port | `localhost:5432` | No |
| `PG_USERNAME` | PostgreSQL username | `postgres` | No |
| `PG_PASSWORD` | PostgreSQL password | `postgres` | No |
| `PG_DATABASE` | PostgreSQL database name | `probod` | No |
| `PG_POOL_SIZE` | Maximum number of connections in the database pool | `100` | No |
## Authentication
### User Authentication
| Variable | Description | Default Value | Required |
| -------------------------------- | --------------------------------------------- | --------------- | -------- |
| `AUTH_DISABLE_SIGNUP` | Disable user self-registration | `false` | No |
| `AUTH_INVITATION_TOKEN_VALIDITY` | Invitation token validity duration in seconds | `3600` (1 hour) | No |
### Authentication Cookies
| Variable | Description | Default Value | Required |
| ---------------------- | -------------------------------------------------- | ------------- | -------- |
| `AUTH_COOKIE_NAME` | Name of the session cookie | `SSID` | No |
| `AUTH_COOKIE_DOMAIN` | Domain for the session cookie | `localhost` | No |
| `AUTH_COOKIE_SECRET` | Secret key for signing session cookies (32+ bytes) | - | **Yes** |
| `AUTH_COOKIE_DURATION` | Session cookie validity duration in hours | `24` | No |
| `AUTH_COOKIE_SECURE` | Set Secure flag on cookies (use false for HTTP) | `true` | No |
### Password Security
| Variable | Description | Default Value | Required |
| -------------------------- | ---------------------------------------------------- | ------------- | -------- |
| `AUTH_PASSWORD_PEPPER` | Secret pepper value for password hashing (32+ bytes) | - | **Yes** |
| `AUTH_PASSWORD_ITERATIONS` | Number of PBKDF2 iterations for password hashing | `1000000` | No |
## Trust Center Authentication
| Variable | Description | Default Value | Required |
| -------------------------------- | ------------------------------------------------------ | ----------------------- | -------- |
| `TRUST_AUTH_COOKIE_NAME` | Name of the trust center token cookie | `TCT` | No |
| `TRUST_AUTH_COOKIE_DOMAIN` | Domain for the trust center cookie | `localhost` | No |
| `TRUST_AUTH_COOKIE_DURATION` | Trust center cookie validity duration in hours | `24` | No |
| `TRUST_AUTH_TOKEN_DURATION` | Trust center access token validity duration in hours | `168` (7 days) | No |
| `TRUST_AUTH_REPORT_URL_DURATION` | Validity duration for report URLs in minutes | `15` | No |
| `TRUST_AUTH_TOKEN_SECRET` | Secret key for signing trust center tokens (32+ bytes) | - | **Yes** |
| `TRUST_AUTH_SCOPE` | OAuth2 scope for trust center access | `trust_center_readonly` | No |
| `TRUST_AUTH_TOKEN_TYPE` | Token type identifier for trust center tokens | `trust_center_access` | No |
## AWS / S3 Storage
| Variable | Description | Default Value | Required |
| ----------------------- | -------------------------------------------------------- | ------------- | -------- |
| `AWS_REGION` | AWS region for S3 storage | `us-east-1` | No |
| `AWS_BUCKET` | S3 bucket name for file storage | `probod` | No |
| `AWS_ACCESS_KEY_ID` | AWS access key ID (leave empty for IAM role) | - | No |
| `AWS_SECRET_ACCESS_KEY` | AWS secret access key (leave empty for IAM role) | - | No |
| `AWS_ENDPOINT` | Custom S3 endpoint (for MinIO or S3-compatible services) | - | No |
## Notifications
### Email (SMTP)
| Variable | Description | Default Value | Required |
| --------------------- | ---------------------------------------------- | ------------------------------------ | -------- |
| `MAILER_SENDER_NAME` | Display name for outgoing emails | `Probo` | No |
| `MAILER_SENDER_EMAIL` | Email address for outgoing emails | `no-reply@notification.getprobo.com` | No |
| `SMTP_ADDR` | SMTP server address and port | `localhost:1025` | No |
| `SMTP_TLS_REQUIRED` | Require TLS for SMTP connections | `false` | No |
| `MAILER_INTERVAL` | Interval in seconds for processing email queue | `60` | No |
### Slack
| Variable | Description | Default Value | Required |
| ----------------------- | ----------------------------------------------------------- | ------------- | -------- |
| `SLACK_SENDER_INTERVAL` | Interval in seconds for processing Slack notification queue | `60` | No |
## OpenAI Integration
| Variable | Description | Default Value | Required |
| -------------------- | ------------------------------------------------------ | ------------- | -------- |
| `OPENAI_API_KEY` | OpenAI API key for AI-powered features | - | No |
| `OPENAI_TEMPERATURE` | Temperature parameter for OpenAI completions (0.0-2.0) | `0.1` | No |
| `OPENAI_MODEL_NAME` | OpenAI model name to use | `gpt-4o` | No |
## SAML Authentication
| Variable | Description | Default Value | Required |
| ------------------------------- | ----------------------------------------------------------------- |------------------| -------- |
| `SAML_SESSION_DURATION` | SAML session validity duration in seconds | `604800` (7 days) | No |
| `SAML_CLEANUP_INTERVAL_SECONDS` | Interval in seconds for cleaning up expired SAML sessions (0=off) | `0` (disabled) | No |
| `SAML_CERTIFICATE` | SAML service provider certificate in PEM format | `autogenerated` | No |
| `SAML_PRIVATE_KEY` | SAML service provider private key in PEM format | `autogenerated` | No |
## Custom Domains
| Variable | Description | Default Value | Required |
| ----------------------------------- | ----------------------------------------------------- | --------------------- | -------- |
| `CUSTOM_DOMAINS_RENEWAL_INTERVAL` | Interval in seconds for checking certificate renewals | `3600` (1 hour) | No |
| `CUSTOM_DOMAINS_PROVISION_INTERVAL` | Interval in seconds for provisioning new domains | `30` | No |
| `CUSTOM_DOMAINS_CNAME_TARGET` | CNAME target for custom domains | `custom.getprobo.com` | No |
### ACME / Let's Encrypt
| Variable | Description | Default Value | Required |
| ---------------- | --------------------------------------------------------------- | ------------------------------------------------ | -------- |
| `ACME_DIRECTORY` | ACME directory URL for certificate issuance | `https://acme-v02.api.letsencrypt.org/directory` | No |
| `ACME_EMAIL` | Email address for ACME account registration | `admin@getprobo.com` | No |
| `ACME_KEY_TYPE` | Key type for ACME certificates (RSA2048, RSA4096, EC256, EC384) | `EC256` | No |
| `ACME_ROOT_CA` | Custom root CA certificate (PEM format) | - | No |
## Connectors
### Slack Connector (OAuth2)
These variables are only used if `CONNECTOR_SLACK_CLIENT_ID` is set.
| Variable | Description | Default Value | Required |
| -------------------------------- | ------------------------------------------------- | ----------------------------------------------------------- | -------------------------- |
| `CONNECTOR_SLACK_CLIENT_ID` | Slack OAuth2 app client ID | - | No |
| `CONNECTOR_SLACK_CLIENT_SECRET` | Slack OAuth2 app client secret | - | **Yes** (if client ID set) |
| `CONNECTOR_SLACK_REDIRECT_URI` | OAuth2 redirect URI for Slack connector | `https://localhost:8080/api/console/v1/connectors/complete` | No |
| `CONNECTOR_SLACK_AUTH_URL` | Slack OAuth2 authorization endpoint | `https://slack.com/oauth/v2/authorize` | No |
| `CONNECTOR_SLACK_TOKEN_URL` | Slack OAuth2 token endpoint | `https://slack.com/api/oauth.v2.access` | No |
| `CONNECTOR_SLACK_SIGNING_SECRET` | Slack app signing secret for webhook verification | - | **Yes** (if client ID set) |
## Security Best Practices
### Required Secrets
The following environment variables are **required** and must be set to secure random values in production:
1. `PROBOD_ENCRYPTION_KEY` - Generate with: `openssl rand -base64 32`
2. `AUTH_COOKIE_SECRET` - Generate with: `openssl rand -base64 32`
3. `AUTH_PASSWORD_PEPPER` - Generate with: `openssl rand -base64 32`
4. `TRUST_AUTH_TOKEN_SECRET` - Generate with: `openssl rand -base64 32`
### Secret Generation Example
```bash
# Generate all required secrets
export PROBOD_ENCRYPTION_KEY=$(openssl rand -base64 32)
export AUTH_COOKIE_SECRET=$(openssl rand -base64 32)
export AUTH_PASSWORD_PEPPER=$(openssl rand -base64 32)
export TRUST_AUTH_TOKEN_SECRET=$(openssl rand -base64 32)
echo "PROBOD_ENCRYPTION_KEY=$PROBOD_ENCRYPTION_KEY"
echo "AUTH_COOKIE_SECRET=$AUTH_COOKIE_SECRET"
echo "AUTH_PASSWORD_PEPPER=$AUTH_PASSWORD_PEPPER"
echo "TRUST_AUTH_TOKEN_SECRET=$TRUST_AUTH_TOKEN_SECRET"
```
## Configuration Priority
The entrypoint script follows this priority order:
1. If `CONFIG_FILE` exists (e.g., mounted from ConfigMap/volume), use it as-is
2. Otherwise, generate config file from environment variables
3. Environment variables use provided values or fall back to defaults
4. Script fails if required variables are missing (marked with `:?` in bash)

784
docs/DOCKER_SERVICES.md
View File

@@ -1,784 +0,0 @@
# Docker Services Documentation
This document describes all services used in the Docker Compose development environment (`compose.yaml`) and their purposes in the Probo application.
## Overview
The development stack includes 10 services organized into the following categories:
- **Core Services**: PostgreSQL, MinIO, Chrome
- **Observability Stack**: Grafana, Prometheus, Loki, Tempo
- **Development Tools**: Mailpit
- **ACME Testing**: Pebble, Pebble Challenge Test Server
## Core Services
### PostgreSQL
**Image**: `postgres:17.4`
**Port**: `5432`
**Purpose**: Database for storing all compliance data
PostgreSQL is the relational database that stores:
- User accounts and organizations
- Compliance frameworks and controls
- Audit trails and evidence
- Documents and policies
- Risk assessments
- Vendor information
(This list is not exhaustive and can be extended as the application grows.)
#### Configuration
```yaml
Environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
Command:
postgres -c "shared_buffers=4GB"
-c "max_connections=200"
-c "log_statement=all"
```
#### Database Setup
On first startup, initialization scripts from `./compose/postgres/` are executed. The database `probod` is created automatically by the application's migration system.
#### Connection Details
- **Host**: `localhost` (or `postgres` from within containers)
- **Port**: `5432`
- **Database**: `probod`
- **Username**: `postgres` / `probod`
- **Password**: `postgres`
#### Access
```bash
# Using psql from host
psql -h localhost -U postgres -d probod
# Using make command
make psql
# From within Docker
docker compose exec postgres psql -U postgres -d probod
```
#### Performance Settings
- **shared_buffers**: 4GB - Memory for caching data
- **max_connections**: 200 - Maximum concurrent connections
- **log_statement**: all - Logs all SQL statements for debugging
---
### MinIO
**Image**: `quay.io/minio/minio`
**Ports**: `9000` (API), `9001` (Console)
**Purpose**: S3-compatible object storage for files and documents
MinIO provides local S3-compatible storage for development, storing:
- Uploaded evidence files
- Generated PDF reports
- Document attachments
- Policy documents
- Exported data
#### Configuration
```yaml
Environment:
MINIO_ROOT_USER: probod
MINIO_ROOT_PASSWORD: thisisnotasecret
Command:
mkdir -p /var/lib/minio/probod &&
minio server --json --console-address :9001 /var/lib/minio
```
#### Bucket Setup
The bucket `probod` is created automatically on startup via the startup command.
#### Access
**Console UI**: http://localhost:9001
- **Username**: `probod`
- **Password**: `thisisnotasecret`
**API Endpoint**: http://localhost:9000
#### S3 Configuration in Probo
```yaml
aws:
region: us-east-1
bucket: probod
access-key-id: probod
secret-access-key: thisisnotasecret
endpoint: http://127.0.0.1:9000
```
#### CLI Access
```bash
# Using AWS CLI
aws --endpoint-url http://localhost:9000 s3 ls s3://probod/
# Using MinIO Client
mc alias set local http://localhost:9000 probod thisisnotasecret
mc ls local/probod
```
---
### Chrome Headless
**Image**: `chromedp/headless-shell:140.0.7259.2`
**Port**: `9222`
**Purpose**: Headless browser for PDF generation and document rendering
Chrome provides browser automation capabilities via the Chrome DevTools Protocol, used for:
- Generating PDF reports from HTML templates
- Rendering documents for preview
- Converting web content to PDF format
#### Configuration
```yaml
Command:
--headless
--disable-gpu
--disable-dev-shm-usage
--hide-scrollbars
--mute-audio
--no-default-browser-check
--no-first-run
--disable-background-networking
--disable-background-timer-throttling
--disable-extensions
```
#### Chrome DevTools Protocol
The service exposes the Chrome DevTools Protocol on port 9222, which Probo uses through the `chromedp` Go library.
#### Probo Configuration
```yaml
probod:
chrome-dp-addr: "localhost:9222"
```
#### Testing
```bash
# Check Chrome version
curl http://localhost:9222/json/version
# List available tabs/targets
curl http://localhost:9222/json/list
```
---
## Observability Stack
The observability stack provides comprehensive monitoring, logging, and tracing for development and debugging.
### Grafana
**Image**: `grafana/grafana:latest`
**Port**: `3001`
**Purpose**: Visualization and dashboards for metrics, logs, and traces
Grafana provides a unified interface for:
- Visualizing Prometheus metrics
- Querying and analyzing logs from Loki
- Viewing distributed traces from Tempo
- Creating custom dashboards
#### Configuration
```yaml
Environment:
GF_AUTH_ANONYMOUS_ENABLED: true
GF_AUTH_ANONYMOUS_ORG_ROLE: Admin
GF_AUTH_DISABLE_LOGIN_FORM: true
GF_USERS_DEFAULT_THEME: light
```
#### Access
**Web UI**: http://localhost:3001
No login required - anonymous access is enabled with Admin role for development.
#### Data Sources
Grafana is pre-configured with:
- **Prometheus** - Metrics at http://prometheus:9191
- **Loki** - Logs at http://loki:3100
- **Tempo** - Traces at http://tempo:4317
Configuration files are in `./compose/grafana/provisioning/`
#### Common Queries
**Metrics (Prometheus)**:
```promql
# Request rate
rate(http_requests_total[5m])
# Error rate
rate(http_requests_total{status=~"5.."}[5m])
```
**Logs (Loki)**:
```logql
{job="probod"} |= "error"
{job="probod"} | json | level="error"
```
---
### Prometheus
**Image**: `prom/prometheus:latest`
**Port**: `9191`
**Purpose**: Metrics collection and storage
Prometheus scrapes and stores time-series metrics from Probo, including:
- HTTP request metrics (rate, duration, status codes)
- Database query metrics
- Business metrics (users, organizations, controls)
- Go runtime metrics (goroutines, memory, GC)
#### Configuration
```yaml
Command:
--config.file=/etc/prometheus/prometheus.yml
--storage.tsdb.path=/prometheus
--web.console.libraries=/etc/prometheus/console_libraries
--web.console.templates=/etc/prometheus/consoles
--web.enable-lifecycle
--web.enable-remote-write-receiver
--web.listen-address=:9191
```
#### Scrape Configuration
Configuration file: `./compose/prometheus/prometheus.yaml`
```yaml
scrape_configs:
- job_name: 'probod'
scrape_interval: 15s
static_configs:
- targets: ['host.docker.internal:8081']
```
#### Access
**Web UI**: http://localhost:9191
#### Metrics Endpoint
Probo exposes metrics at: http://localhost:8081/metrics
#### Useful Queries
```promql
# Total requests
sum(rate(http_requests_total[5m]))
# P95 latency
histogram_quantile(0.95, rate(http_request_duration_seconds_bucket[5m]))
# Active database connections
pg_connections_active
```
---
### Loki
**Image**: `grafana/loki:latest`
**Port**: `3100`
**Purpose**: Log aggregation and querying
Loki collects and indexes logs from Probo, providing:
- Centralized log storage
- Efficient log querying
- Label-based log filtering
- Integration with Grafana for visualization
#### Configuration
Uses default configuration from `/etc/loki/local-config.yaml` in the container.
#### Log Ingestion
Probo sends structured JSON logs to stdout, which can be collected by:
- Docker logging drivers
- Promtail (Loki's log shipper)
- Direct HTTP API calls
#### Access
**API**: http://localhost:3100
#### Query Examples
```bash
# Query logs via API
curl -G -s "http://localhost:3100/loki/api/v1/query_range" \
--data-urlencode 'query={job="probod"}' \
--data-urlencode 'limit=100'
# Query in Grafana
{job="probod"} |= "error" | json
```
#### Log Format
Probo outputs structured JSON logs:
```json
{
"level": "info",
"time": "2024-01-01T12:00:00Z",
"caller": "server/server.go:123",
"msg": "request completed",
"method": "GET",
"path": "/api/v1/controls",
"status": 200,
"duration": 45.2,
"trace_id": "abc123"
}
```
---
### Tempo
**Image**: `grafana/tempo:latest`
**Port**: `4317` (OTLP gRPC)
**Purpose**: Distributed tracing backend
Tempo stores and queries distributed traces, providing:
- End-to-end request tracing
- Service dependency visualization
- Performance bottleneck identification
- Trace-to-log correlation
#### Configuration
Configuration file: `./compose/tempo/tempo.yaml`
#### OpenTelemetry Integration
Probo uses OpenTelemetry instrumentation to send traces to Tempo via OTLP/gRPC protocol.
#### Probo Configuration
```yaml
unit:
tracing:
addr: "localhost:4317"
max-batch-size: 512
batch-timeout: 5
export-timeout: 30
max-queue-size: 2048
```
#### Access
Traces are viewed through Grafana's Explore interface at http://localhost:3001
#### Trace Context
Each trace includes:
- **Trace ID**: Unique identifier for the entire request
- **Span ID**: Identifier for each operation
- **Duration**: How long each operation took
- **Tags**: Metadata (HTTP method, status, user ID, etc.)
#### Correlation
Traces are automatically correlated with:
- **Logs**: Trace ID is included in log entries
- **Metrics**: Exemplars link metrics to traces
---
## Development Tools
### Mailpit
**Image**: `axllent/mailpit:latest`
**Ports**: `1025` (SMTP), `8025` (Web UI)
**Purpose**: Email testing and debugging
Mailpit is an email testing tool that captures all outgoing emails without actually sending them, allowing you to:
- Test email functionality locally
- Preview email templates
- Debug email content and formatting
- Verify email delivery logic
#### Configuration
```yaml
Environment:
MP_DISABLE_VERSION_CHECK: true
MP_VERBOSE: false
MP_SMTP_AUTH_ACCEPT_ANY: true
MP_ENABLE_PROMETHEUS: true
MP_SMTP_AUTH_ALLOW_INSECURE: true
```
#### Probo Configuration
```yaml
mailer:
sender-name: "Probo"
sender-email: "no-reply@notification.getprobo.com"
smtp:
addr: "localhost:1025"
tls-required: false
```
#### Access
**Web UI**: http://localhost:8025
#### Features
- View all captured emails in a web interface
- Search and filter emails
- View HTML and plain text versions
- Check email headers and attachments
- API access for automated testing
#### Email Types Sent by Probo
- User invitation emails
- Password reset emails
- Audit notification emails
- Report delivery emails
- Task assignment notifications
#### API Access
```bash
# List all messages
curl http://localhost:8025/api/v1/messages
# Get specific message
curl http://localhost:8025/api/v1/message/{id}
```
---
## ACME Testing Services
These services enable local testing of Let's Encrypt certificate provisioning for custom domains.
### Pebble
**Image**: `letsencrypt/pebble:latest`
**Ports**: `14000` (ACME), `15000` (Management)
**Purpose**: ACME protocol test server for Let's Encrypt simulation
Pebble is a small ACME test server that mimics Let's Encrypt, allowing:
- Local testing of ACME certificate provisioning
- Validation of custom domain SSL setup
- Testing certificate renewal logic
- Fast iteration without rate limits
#### Configuration
```yaml
Environment:
PEBBLE_VA_NOSLEEP: "1" # Fast validation
PEBBLE_WFE_NONCEREJECT: "0" # Allow reused nonces
PEBBLE_VA_ALWAYS_VALID: "1" # Skip actual validation
Command:
pebble -config /test/config/pebble-config.json
-dnsserver 127.0.0.1:8053
```
Configuration file: `./compose/pebble/pebble-config.json`
#### Probo Configuration
```yaml
custom-domains:
acme:
directory: "https://localhost:14000/dir"
email: "admin@getprobo.com"
key-type: "EC256"
insecure-tls: true
```
#### ACME Endpoints
- **Directory**: https://localhost:14000/dir
- **Management API**: http://localhost:15000
#### Certificates
Pebble issues certificates signed by its own CA. The root CA certificate is available at:
`./compose/pebble/certs/rootCA.pem`
This certificate is generated by `mkcert` and must be trusted locally for HTTPS to work.
#### Testing Custom Domains
1. Request certificate from Pebble
2. Complete HTTP-01 or DNS-01 challenge
3. Receive certificate (valid for 90 days)
4. Test certificate renewal
---
### Pebble Challenge Test Server
**Image**: `letsencrypt/pebble-challtestsrv:latest`
**Ports**: `8055` (HTTP-01), `8053` (DNS), `8056` (Management)
**Purpose**: Challenge validation server for ACME testing
This service handles ACME challenge validation:
- **HTTP-01**: Serves challenge responses at `/.well-known/acme-challenge/`
- **DNS-01**: Responds to DNS TXT record queries
- **Management API**: Control challenge responses
#### Configuration
```yaml
Command:
pebble-challtestsrv -dns01 ":8053"
-http01 ":8055"
-management ":8056"
```
#### How It Works
1. Probo requests certificate from Pebble
2. Pebble creates a challenge
3. Probo provisions the challenge response
4. Pebble validates by querying this server
5. Certificate is issued if validation succeeds
#### Management API
```bash
# Add HTTP-01 challenge response
curl -X POST http://localhost:8056/add-http01 \
-d '{"token":"abc", "content":"xyz"}'
# Add DNS-01 TXT record
curl -X POST http://localhost:8056/set-txt \
-d '{"host":"_acme-challenge.example.com", "value":"abc123"}'
```
---
## Service Dependencies
```
probo (application)
├── depends on: postgres (database)
├── depends on: minio (file storage)
├── depends on: chrome (PDF generation)
├── sends metrics to: prometheus
├── sends logs to: loki
├── sends traces to: tempo
├── sends emails to: mailpit
└── uses for ACME: pebble + pebble-challtestsrv
grafana
├── queries: prometheus (metrics)
├── queries: loki (logs)
└── queries: tempo (traces)
```
## Starting and Stopping Services
### Start All Services
```bash
make stack-up
# or
docker compose up -d
```
### Stop All Services
```bash
make stack-down
# or
docker compose down
```
### View Running Services
```bash
make stack-ps
# or
docker compose ps
```
### View Logs
```bash
# All services
docker compose logs -f
# Specific service
docker compose logs -f postgres
docker compose logs -f minio
```
### Restart Service
```bash
docker compose restart postgres
```
## Volumes and Data Persistence
The following volumes persist data across container restarts:
- `postgres-data` - PostgreSQL database files
- `minio-data` - MinIO object storage
- `grafana-data` - Grafana dashboards and settings
- `prometheus-data` - Prometheus metrics database
- `tempo-data` - Tempo trace storage
### Clearing Data
```bash
# Remove all volumes (WARNING: deletes all data)
docker compose down -v
# Remove specific volume
docker volume rm probo_postgres-data
```
## Network Configuration
All services run on a default Docker Compose network and can communicate using service names:
- From Probo: `postgres:5432`, `minio:9000`, `chrome:9222`
- From Grafana: `prometheus:9191`, `loki:3100`, `tempo:4317`
## Port Summary
| Service | Port(s) | Purpose |
|---------|---------|---------|
| PostgreSQL | 5432 | Database access |
| MinIO | 9000, 9001 | S3 API, Console UI |
| Chrome | 9222 | DevTools Protocol |
| Grafana | 3001 | Web UI |
| Prometheus | 9191 | Metrics API, Web UI |
| Loki | 3100 | Log ingestion API |
| Tempo | 4317 | OTLP trace ingestion |
| Mailpit | 1025, 8025 | SMTP, Web UI |
| Pebble | 14000, 15000 | ACME API, Management |
| Pebble ChalTest | 8053, 8055, 8056 | DNS, HTTP-01, Management |
## Resource Requirements
Minimum recommended resources for development:
- **CPU**: 4 cores
- **Memory**: 8GB RAM
- **Disk**: 20GB free space
Individual service resources:
- PostgreSQL: 2GB RAM (shared_buffers=4GB)
- MinIO: 512MB RAM
- Chrome: 1GB RAM
- Observability stack: ~2GB RAM combined
## Troubleshooting
### PostgreSQL Connection Issues
```bash
# Check if PostgreSQL is running
docker compose ps postgres
# View PostgreSQL logs
docker compose logs postgres
# Test connection
psql -h localhost -U postgres -c "SELECT 1"
```
### MinIO Access Issues
```bash
# Check MinIO health
curl http://localhost:9000/minio/health/live
# List buckets
aws --endpoint-url http://localhost:9000 s3 ls
```
### Chrome Not Responding
```bash
# Check Chrome status
curl http://localhost:9222/json/version
# Restart Chrome
docker compose restart chrome
```
### Observability Stack Issues
```bash
# Check Prometheus targets
curl http://localhost:9191/api/v1/targets
# Check Loki ready status
curl http://localhost:3100/ready
# Check Tempo ready status
curl http://localhost:4317
```
## Security Notes
⚠️ **Development Only**: This Docker Compose setup is for development and should NOT be used in production.
- Default passwords are used (change in production)
- Anonymous access is enabled in Grafana
- TLS is disabled for many services
- No network isolation
- Insecure ACME validation
For production deployment, use:
- Managed database services (AWS RDS, GCP Cloud SQL)
- Managed object storage (AWS S3, GCS)
- Proper authentication and TLS everywhere
- Network policies and firewalls
## Additional Resources
- [PostgreSQL Documentation](https://www.postgresql.org/docs/)
- [MinIO Documentation](https://min.io/docs/)
- [Chrome DevTools Protocol](https://chromedevtools.github.io/devtools-protocol/)
- [Grafana Documentation](https://grafana.com/docs/)
- [Prometheus Documentation](https://prometheus.io/docs/)
- [Loki Documentation](https://grafana.com/docs/loki/)
- [Tempo Documentation](https://grafana.com/docs/tempo/)
- [OpenTelemetry Documentation](https://opentelemetry.io/docs/)
- [Pebble GitHub](https://github.com/letsencrypt/pebble)

451
docs/INSTALLATION.md
View File

@@ -1,451 +0,0 @@
# Probo Installation Guide
This document provides comprehensive installation instructions for the Probo compliance management platform daemon (`probod`).
## Installation Methods
Probo can be deployed using either the official Docker image or pre-compiled binaries available through GitHub releases.
## Docker Installation
The official Docker images are available on GitHub Container Registry and support multiple architectures:
- **Multi-architecture image**: `ghcr.io/getprobo/probo:latest`
- **AMD64 (x86_64)**: `ghcr.io/getprobo/probo:latest-amd64`
- **ARM64**: `ghcr.io/getprobo/probo:latest-arm64`
### Basic Docker Setup
To run Probo using Docker:
```bash
docker run -d \
--name probod \
-p 8080:8080 \
-v /path/to/config.yaml:/etc/probod/config.yaml \
ghcr.io/getprobo/probo:latest
```
### Docker Compose Setup
For a complete setup with dependencies, you can use our `compose.prod.yml` Docker Compose file:
You can either provide environment variables directly in the docker-compose file or use a config file mounted as a volume.
#### With Environment Variables
You can find environment variable options in the [docker environment variables](./DOCKER_ENVIRONMENT_VARIABLES.md).
```yaml
services:
probo:
image: "ghcr.io/getprobo/probo:latest"
environment:
# Required secrets (use secure values in production)
PROBOD_ENCRYPTION_KEY: "thisisnotasecretAAAAAAAAAAAAAAAAAAAAAAAAAAA="
AUTH_COOKIE_SECRET: "this-is-a-secure-secret-for-cookie-signing-at-least-32-bytes"
AUTH_PASSWORD_PEPPER: "this-is-a-secure-pepper-for-password-hashing-at-least-32-bytes"
TRUST_AUTH_TOKEN_SECRET: "this-is-a-secure-secret-for-trust-token-signing-at-least-32-bytes"
# Application settings
PROBOD_BASE_URL: "http://localhost:8080"
API_ADDR: "localhost:8080"
API_CORS_ALLOWED_ORIGINS: "http://localhost:8080"
# PostgreSQL database
PG_ADDR: "postgres:5432"
PG_USERNAME: "postgres"
PG_PASSWORD: "postgres"
PG_DATABASE: "probod"
PG_POOL_SIZE: "100"
# AWS/MinIO S3 storage
AWS_REGION: "us-east-1"
AWS_BUCKET: "probod"
AWS_ACCESS_KEY_ID: "probod"
AWS_SECRET_ACCESS_KEY: "thisisnotasecret"
AWS_ENDPOINT: "http://minio:9000"
# Observability - Metrics & Tracing
METRICS_ADDR: "probo:8081"
TRACING_ADDR: ""
# Email notifications
SMTP_ADDR: "your.smtp.server:587"
SMTP_TLS_REQUIRED: "false"
MAILER_SENDER_NAME: "Probo"
MAILER_SENDER_EMAIL: "no-reply@notification.getprobo.com"
# Chrome for PDF generation
CHROME_DP_ADDR: "chrome:9222"
ports:
- "8080:8080"
- "8081:8081"
- "8443:8443"
volumes:
- "probo-data:/data"
depends_on:
- postgres
- minio
- chrome
postgres:
image: "postgres:17.4"
shm_size: "1g"
command: >
postgres -c "shared_buffers=4GB"
-c "max_connections=200"
-c "log_statement=all"
ports:
- "5432:5432"
volumes:
- "./compose/postgres:/docker-entrypoint-initdb.d:ro"
- "postgres-data:/var/lib/postgresql/data:rw"
environment:
POSTGRES_USER: "postgres"
POSTGRES_PASSWORD: "postgres"
minio:
image: "quay.io/minio/minio"
entrypoint: "sh"
command: |
-c 'mkdir -p /var/lib/minio/probod && minio server --json --console-address :9001 /var/lib/minio'
ports:
- "9000:9000"
- "9001:9001"
volumes:
- "minio-data:/var/lib/minio:rw"
environment:
MINIO_ROOT_USER: "probod"
MINIO_ROOT_PASSWORD: "thisisnotasecret"
chrome:
image: "chromedp/headless-shell:140.0.7259.2"
ports:
- "9222:9222"
command:
- "--headless"
- "--disable-gpu"
- "--disable-dev-shm-usage"
- "--hide-scrollbars"
- "--mute-audio"
- "--no-default-browser-check"
- "--no-first-run"
- "--disable-background-networking"
- "--disable-background-timer-throttling"
- "--disable-extensions"
volumes:
probo-data:
postgres-data:
minio-data:
```
#### With mounted Config File
You can find an example configuration file [here](../cfg/dev.yaml) and modify it as needed.
```yaml
services:
probo:
image: "ghcr.io/getprobo/probo:latest"
ports:
- "8080:8080"
- "8081:8081"
- "8443:8443"
environment:
- PROBOD_CONFIG=/etc/probod/config.yaml
volumes:
- "probo-data:/data"
- "./cfg/dev.yaml:/etc/probod/config.yaml:ro"
depends_on:
- postgres
- minio
- chrome
postgres:
image: "postgres:17.4"
shm_size: "1g"
command: >
postgres -c "shared_buffers=4GB"
-c "max_connections=200"
-c "log_statement=all"
ports:
- "5432:5432"
volumes:
- "./compose/postgres:/docker-entrypoint-initdb.d:ro"
- "postgres-data:/var/lib/postgresql/data:rw"
environment:
POSTGRES_USER: "postgres"
POSTGRES_PASSWORD: "postgres"
minio:
image: "quay.io/minio/minio"
entrypoint: "sh"
command: |
-c 'mkdir -p /var/lib/minio/probod && minio server --json --console-address :9001 /var/lib/minio'
ports:
- "9000:9000"
- "9001:9001"
volumes:
- "minio-data:/var/lib/minio:rw"
environment:
MINIO_ROOT_USER: "probod"
MINIO_ROOT_PASSWORD: "thisisnotasecret"
chrome:
image: "chromedp/headless-shell:140.0.7259.2"
ports:
- "9222:9222"
command:
- "--headless"
- "--disable-gpu"
- "--disable-dev-shm-usage"
- "--hide-scrollbars"
- "--mute-audio"
- "--no-default-browser-check"
- "--no-first-run"
- "--disable-background-networking"
- "--disable-background-timer-throttling"
- "--disable-extensions"
volumes:
probo-data:
postgres-data:
minio-data:
```
### Docker Architecture Support
The Docker images support the following architectures:
- **linux/amd64** (x86_64) - Standard 64-bit Intel/AMD processors
- **linux/arm64** - ARM 64-bit processors (Apple Silicon, AWS Graviton, etc.)
Multi-architecture images automatically select the appropriate variant for your platform.
## Binary Installation
Pre-compiled binaries are available for download from the [GitHub releases page](https://github.com/getprobo/probo/releases).
### Supported Platforms
The following platforms are officially supported:
- **Windows**: AMD64 (x86_64)
- **macOS**: AMD64 (x86_64) and ARM64 (Apple Silicon)
- **Linux**: AMD64 (x86_64) and ARM64 (via Docker)
If your specific platform is not available, please contact us, and we may be able to add support for additional architectures.
### Installation Steps
1. **Download the Binary**
Visit the [GitHub releases page](https://github.com/getprobo/probo/releases) and download the appropriate archive for your platform:
- **Windows**: `probod_Windows_x86_64.zip`
- **macOS (Intel)**: `probod_Darwin_x86_64.tar.gz`
- **macOS (Apple Silicon)**: `probod_Darwin_arm64.tar.gz`
2. **Extract the Archive**
```bash
# For tar.gz files (macOS/Linux)
tar -xzf probod_Darwin_x86_64.tar.gz
# For zip files (Windows)
# Use your preferred extraction tool
```
3. **Install the Binary**
**macOS/Linux:**
```bash
# Move to a directory in your PATH
sudo mv probod /usr/local/bin/
# Make executable (if not already)
sudo chmod +x /usr/local/bin/probod
```
**Windows:**
```cmd
# Move probod.exe to a directory in your PATH
# Or add the current directory to your PATH environment variable
```
4. **Verify Installation**
```bash
probod --version
```
### Running the Binary
Once installed, you can run Probo with a configuration file:
```bash
# Using the default configuration location
probod --config /etc/probod/config.yaml
# Or specify a custom configuration file
probod --config ./my-config.yaml
```
## System Requirements
### Minimum Requirements
- **CPU**: 1 core, 2 GHz
- **Memory**: 1 GB RAM
- **Storage**: 10 GB available space
- **Network**: Internet connectivity for external integrations
### Recommended Requirements
- **CPU**: 2+ cores, 2.4 GHz
- **Memory**: 4 GB RAM
- **Storage**: 50 GB available space (SSD preferred)
- **Network**: Stable internet connection
### Dependencies
Probo requires the following external services:
1. **PostgreSQL Database** (version 12 or higher)
2. **S3-Compatible Storage** (AWS S3, MinIO, etc.)
3. **Chrome/Chromium** (for PDF generation via Chrome DevTools Protocol)
Optional dependencies:
- **SMTP Server** (for email notifications)
- **OpenAI API** (for AI-powered features)
## Quick Start
### 1. Database Setup
Create a PostgreSQL database for Probo:
```sql
CREATE DATABASE probod;
CREATE USER postgres WITH PASSWORD 'your_secure_password';
GRANT ALL PRIVILEGES ON DATABASE probod TO postgres;
```
### 2. Configuration
Create a basic configuration file (`config.yaml`):
```yaml
probod:
hostname: "localhost:8080"
encryption-key: "your-base64-encoded-encryption-key"
pg:
addr: "localhost:5432"
username: "postgres"
password: "your_secure_password"
database: "probod"
aws:
region: "us-east-1"
bucket: "probod"
access-key-id: "your-access-key"
secret-access-key: "your-secret-key"
endpoint: "http://localhost:9000" # For MinIO
```
### 3. Start the Service
```bash
# Using Docker
docker run -d \
--name probod \
-p 8080:8080 \
-v ./config.yaml:/etc/probod/config.yaml \
ghcr.io/getprobo/probo:latest
# Using Binary
probod --config config.yaml
```
### 4. Access the Application
Open your web browser and navigate to `http://localhost:8080` to access the Probo web interface.
## Production Deployment
### Security Considerations
1. **Use strong, unique secrets** for all authentication components
2. **Enable TLS** for all external communications
3. **Use managed database services** with encryption at rest
4. **Implement proper monitoring** and logging
5. **Regular security updates** and vulnerability assessments
### Load Balancing
For high-availability deployments, consider using:
- **Reverse Proxy**: Nginx, HAProxy, or cloud load balancers
- **Database Clustering**: PostgreSQL with read replicas
- **File Storage**: Distributed S3-compatible storage
### Monitoring
Probo provides metrics and health checks:
- **Health Check**: `GET /health`
- **Metrics**: Prometheus-compatible metrics endpoint
- **Logging**: Structured JSON logging with configurable levels
## Troubleshooting
### Common Installation Issues
1. **Permission Denied (Binary)**
```bash
chmod +x probod
```
2. **Database Connection Failed**
- Verify database credentials and network connectivity
- Check PostgreSQL is running and accepting connections
3. **Docker Image Pull Failed**
```bash
docker login ghcr.io
docker pull ghcr.io/getprobo/probo:latest
```
4. **Port Already in Use**
```bash
# Find process using port 8080
lsof -i :8080
# Or use a different port in configuration
```
### Getting Help
- **Documentation**: Check the [configuration reference](./CONFIGURATION.md)
- **GitHub Issues**: Report bugs and request features
- **Community**: Join our community discussions
### Log Analysis
Enable debug logging to troubleshoot issues:
```yaml
probod:
# ... other configuration
log-level: debug
```
Check startup logs for configuration validation errors and service initialization issues.

139
docs/README.md
View File

@@ -1,138 +1 @@
# Probod Operations Documentation
This documentation provides operational guidance for deploying, configuring, and managing the Probod compliance management daemon.
## Service Components
Probod requires the following components for operation:
- **probod**: Main service daemon
- **PostgreSQL**: Database backend (version 17+)
- **S3-Compatible Storage**: Object storage (AWS S3, MinIO, etc.)
- **Chrome DevTools Protocol**: PDF generation service
## Deployment Guide
- **[Installation Guide](./INSTALLATION.md)** - Docker and binary deployment procedures
- **[Configuration Reference](./CONFIGURATION.md)** - Complete configuration options and examples
- **[Docker Services](./DOCKER_SERVICES.md)** - Development environment services documentation
## Quick Deployment
### Docker Deployment
```bash
docker run -d \
--name probod \
-p 8080:8080 \
-v /path/to/config.yaml:/etc/probod/config.yaml \
ghcr.io/getprobo/probo:latest
```
### Service Verification
```bash
# Health check
curl http://localhost:8080/health
# Service logs
docker logs probod
```
## System Requirements
### Minimum Requirements
- **CPU**: 1 core, 2 GHz
- **Memory**: 1 GB RAM
- **Storage**: 10 GB available space
- **Database**: PostgreSQL 12+
- **Storage Backend**: S3-compatible object storage
### Recommended for Production
- **CPU**: 2+ cores, 2.4 GHz
- **Memory**: 4 GB RAM
- **Storage**: 50 GB SSD
- **Database**: Managed PostgreSQL with backups
- **Load Balancer**: For high availability deployments
## External Dependencies
### Required Services
- **PostgreSQL Database**: Primary data storage
- **S3-Compatible Storage**: File and document storage (AWS S3, MinIO, etc.)
- **Chrome/Chromium**: PDF generation via DevTools Protocol
### Optional Integrations
- **SMTP Server**: Email notifications
- **OpenAI API**: AI-powered features
- **OAuth2 Providers**: External service integrations (GitHub, Slack, etc.)
## Production Deployment
### High Availability
- Deploy multiple probod instances behind a load balancer
- Use managed PostgreSQL with read replicas
- Configure distributed object storage
- Implement monitoring and alerting
### Security Configuration
- Enable TLS for all external communications
- Use strong, unique secrets for authentication components
- Configure network security groups and firewalls
- Regular security updates and vulnerability scanning
## Operations
### Monitoring
- **Health Check**: `GET /health`
- **Metrics**: Prometheus-compatible metrics at `/metrics` endpoint
- **Tracing**: OpenTelemetry distributed tracing support
- **Logging**: Structured JSON logging with configurable levels
#### Configuration-Based Observability
Probod provides built-in observability configured through the main configuration file:
- **Prometheus Metrics**: Configure `unit.metrics.addr` for metrics endpoint
- **OpenTelemetry Tracing**: Configure `unit.tracing` section for trace collection
- **Structured Logging**: Automatic JSON logging with correlation IDs
#### Configuration Example
```yaml
unit:
metrics:
addr: "0.0.0.0:8081"
tracing:
addr: "tempo:4317"
```
See [Configuration Reference](./CONFIGURATION.md#telemetry-and-observability) for complete details.
### Backup and Recovery
- **Database**: Regular PostgreSQL backups with point-in-time recovery
- **Object Storage**: S3 versioning and cross-region replication
- **Configuration**: Version control all configuration files
### Troubleshooting
Common operational issues:
- Verify database connectivity and credentials
- Check S3 storage access permissions
- Validate configuration file syntax
- Review service logs for startup errors
## Support
- **Repository**: [getprobo/probo](https://github.com/getprobo/probo)
- **Issues**: Report operational problems on GitHub
- **Releases**: [GitHub releases](https://github.com/getprobo/probo/releases)
Move to [getprobo.com/docs](https://wwww.getprobo.com/docs]