Bootstrap basic docs

Signed-off-by: Bryan Frimin <bryan@getprobo.com>
This commit is contained in:
Bryan Frimin
2025-08-11 12:41:56 +04:00
parent d016c10fb7
commit 6c63df0cc6
3 changed files with 1071 additions and 0 deletions

623
docs/CONFIGURATION.md Normal file
View File

@@ -0,0 +1,623 @@
# Probod Configuration Reference
This document provides a comprehensive reference for configuring the Probo compliance management platform daemon (`probod`).
For installation instructions, please refer to the [Installation Guide](./INSTALLATION.md).
## Configuration File Format
Probod uses YAML format for configuration files. The configuration is structured hierarchically with the root key `probod` containing all service-specific settings.
### Basic Configuration Structure
```yaml
unit:
metrics:
addr: "localhost:8081"
tracing:
addr: "localhost:4317"
max-batch-size: 1000
batch-timeout: 10
export-timeout: 10
max-queue-size: 10000
probod:
hostname: "localhost:8080"
encryption-key: "base64-encoded-encryption-key"
chrome-dp-addr: "localhost:9222"
api:
addr: "localhost:8080"
cors:
allowed-origins: ["http://localhost:3000"]
extra-header-fields:
"Custom-Header": "value"
pg:
addr: "localhost:5432"
username: "probod"
password: "probod"
database: "probod"
pool-size: 100
ca-cert-bundle: |
-----BEGIN CERTIFICATE-----
...certificate content...
-----END CERTIFICATE-----
auth:
disable-signup: false
invitation-confirmation-token-validity: 3600
cookie:
name: "SSID"
domain: "localhost"
secret: "cookie-signing-secret"
duration: 24
password:
pepper: "password-hashing-pepper"
iterations: 1000000
trust-auth:
cookie-name: "TCT"
cookie-domain: "localhost"
cookie-duration: 24
token-duration: 168
report-url-duration: 15
token-secret: "trust-token-signing-secret"
scope: "trust_center_readonly"
token-type: "trust_center_access"
aws:
region: "us-east-1"
bucket: "probod"
access-key-id: "access-key"
secret-access-key: "secret-key"
endpoint: "http://127.0.0.1:9000"
mailer:
sender-name: "Probo"
sender-email: "no-reply@notification.getprobo.com"
smtp:
addr: "localhost:1025"
user: "smtp-username"
password: "smtp-password"
tls-required: true
openai:
api-key: "openai-api-key"
temperature: 0.1
model-name: "gpt-4o"
connectors:
- name: "github"
type: "oauth2"
config:
client-id: "github-client-id"
client-secret: "github-client-secret"
redirect-uri: "https://localhost:8080/api/console/v1/connectors/complete"
auth-url: "https://github.com/login/oauth/authorize"
token-url: "https://github.com/login/oauth/access_token"
```
## Telemetry and Observability
Probod includes built-in support for Prometheus metrics and OpenTelemetry tracing. The telemetry configuration is part of the main configuration file structure.
### Telemetry Configuration
The telemetry configuration is specified at the top level of the configuration file under the `unit` section:
```yaml
unit:
metrics:
addr: "localhost:8081"
tracing:
addr: "localhost:8082"
max-batch-size: 1000
batch-timeout: 10
export-timeout: 10
max-queue-size: 10000
probod:
# ... rest of probod configuration
```
### Prometheus Metrics
#### `unit.metrics.addr` (string)
**Default**: Same as API address
Network address where the Prometheus metrics endpoint will be exposed. The metrics are available at `/metrics` on this address.
#### Example Configuration
```yaml
unit:
metrics:
addr: "0.0.0.0:8081" # Expose metrics on port 8081
```
### OpenTelemetry Tracing
#### `unit.tracing.addr` (string)
**Default**: Not configured (tracing disabled)
Network address for the OpenTelemetry trace exporter endpoint.
#### `unit.tracing.max-batch-size` (integer)
**Default**: `512`
Maximum number of spans to batch before exporting.
#### `unit.tracing.batch-timeout` (integer)
**Default**: `5` (seconds)
Maximum time to wait before exporting a batch of spans.
#### `unit.tracing.export-timeout` (integer)
**Default**: `30` (seconds)
Maximum time to wait for span export to complete.
#### `unit.tracing.max-queue-size` (integer)
**Default**: `2048`
Maximum queue size for spans waiting to be exported.
#### Example Configuration
```yaml
unit:
tracing:
addr: "tempo:4317" # OTLP gRPC endpoint
max-batch-size: 1000
batch-timeout: 10
export-timeout: 10
max-queue-size: 10000
```
#### Built-in Instrumentation
Probod automatically instruments:
- Database operations (PostgreSQL queries)
- HTTP requests and responses
- GraphQL operations and resolvers
- S3 storage operations
- PDF generation processes
#### Prometheus Scrape Configuration
Configure Prometheus to scrape metrics from probod:
```yaml
# prometheus.yml
global:
scrape_interval: 15s
scrape_configs:
- job_name: "probod"
scrape_interval: 15s
static_configs:
- targets: ["probod:8081"] # Use metrics addr from config
metrics_path: "/metrics"
```
### Logging
Probod provides automatic structured JSON logging with:
- Request correlation IDs
- Integration with OpenTelemetry trace and span IDs
- Component-specific loggers
- Consistent formatting across all services
## Configuration Sections
### General Settings
#### `hostname` (string)
**Default**: `"localhost:8080"`
The hostname and port where the Probod service will be accessible externally. This setting affects URL generation for redirects and API responses.
#### `encryption-key` (string)
**Required**
Base64-encoded encryption key used for encrypting sensitive data at rest. Must be provided for production deployments.
#### `chrome-dp-addr` (string)
**Default**: `"localhost:9222"`
Address of the Chrome DevTools Protocol endpoint used for PDF generation and document processing.
### API Configuration
#### `api.addr` (string)
**Default**: `"localhost:8080"`
The network address and port where the Probod API server will listen for incoming connections.
#### `api.cors.allowed-origins` (array of strings)
**Default**: `[]`
List of origins allowed for Cross-Origin Resource Sharing (CORS) requests. Required for web applications accessing the API from different domains.
#### `api.extra-header-fields` (map of string to string)
**Default**: `{}`
Additional HTTP headers to include in API responses. Useful for custom security headers or integration requirements.
### Database Configuration
#### `pg.addr` (string)
**Default**: `"localhost:5432"`
PostgreSQL database server address and port.
#### `pg.username` (string)
**Default**: `"probod"`
Database username for authentication.
#### `pg.password` (string)
**Default**: `"probod"`
Database password for authentication.
#### `pg.database` (string)
**Default**: `"probod"`
Name of the PostgreSQL database to connect to.
#### `pg.pool-size` (integer)
**Default**: `100`
Maximum number of database connections in the connection pool.
#### `pg.ca-cert-bundle` (string)
**Optional**
PEM-encoded CA certificate bundle for TLS database connections. Required when connecting to databases with custom or self-signed certificates.
### Authentication Configuration
#### `auth.disable-signup` (boolean)
**Default**: `false`
When set to `true`, disables new user registration through the web interface.
#### `auth.invitation-confirmation-token-validity` (integer)
**Default**: `3600`
Validity period (in seconds) for email invitation confirmation tokens.
#### `auth.cookie.name` (string)
**Default**: `"SSID"`
Name of the HTTP cookie used for session management.
#### `auth.cookie.domain` (string)
**Default**: `"localhost"`
Domain scope for session cookies.
#### `auth.cookie.secret` (string)
**Default**: Auto-generated
Secret key used for signing session cookies. Should be at least 32 bytes for security.
#### `auth.cookie.duration` (integer)
**Default**: `24`
Session cookie lifetime in hours.
#### `auth.password.pepper` (string)
**Default**: Auto-generated
Additional secret value used in password hashing. Should be at least 32 bytes and kept confidential.
#### `auth.password.iterations` (integer)
**Default**: `1000000`
Number of iterations for password hashing algorithm (PBKDF2). Higher values increase security but require more computational resources.
### Trust Center Authentication
#### `trust-auth.cookie-name` (string)
**Default**: `"TCT"`
Name of the HTTP cookie used for trust center access tokens.
#### `trust-auth.cookie-domain` (string)
**Default**: `"localhost"`
Domain scope for trust center cookies.
#### `trust-auth.cookie-duration` (integer)
**Default**: `24`
Trust center cookie lifetime in hours.
#### `trust-auth.token-duration` (integer)
**Default**: `168`
Trust center access token lifetime in hours.
#### `trust-auth.report-url-duration` (integer)
**Default**: `15`
Validity period for generated report URLs in minutes.
#### `trust-auth.token-secret` (string)
**Default**: Auto-generated
Secret key used for signing trust center tokens. Should be at least 32 bytes.
#### `trust-auth.scope` (string)
**Default**: `"trust_center_readonly"`
OAuth2 scope for trust center access.
#### `trust-auth.token-type` (string)
**Default**: `"trust_center_access"`
Type identifier for trust center access tokens.
### AWS Configuration
#### `aws.region` (string)
**Default**: `"us-east-1"`
AWS region for S3 bucket operations.
#### `aws.bucket` (string)
**Default**: `"probod"`
S3 bucket name for file storage.
#### `aws.access-key-id` (string)
**Required**
AWS access key ID for authentication.
#### `aws.secret-access-key` (string)
**Required**
AWS secret access key for authentication.
#### `aws.endpoint` (string)
**Optional**
Custom S3-compatible endpoint URL. Useful for local development with MinIO or other S3-compatible services.
### Email Configuration
#### `mailer.sender-name` (string)
**Default**: `"Probo"`
Display name for outgoing emails.
#### `mailer.sender-email` (string)
**Default**: `"no-reply@notification.getprobo.com"`
Email address used as the sender for outgoing emails.
#### `mailer.smtp.addr` (string)
**Default**: `"localhost:1025"`
SMTP server address and port.
#### `mailer.smtp.user` (string)
**Optional**
Username for SMTP authentication.
#### `mailer.smtp.password` (string)
**Optional**
Password for SMTP authentication.
#### `mailer.smtp.tls-required` (boolean)
**Default**: `false`
Whether TLS encryption is required for SMTP connections.
### OpenAI Integration
#### `openai.api-key` (string)
**Required for AI features**
API key for OpenAI services integration.
#### `openai.temperature` (float)
**Default**: `0.1`
Temperature parameter for AI model responses (0.0 to 1.0). Lower values produce more deterministic outputs.
#### `openai.model-name` (string)
**Default**: `"gpt-4o"`
OpenAI model identifier to use for AI-powered features.
### External Connectors
The `connectors` section defines external service integrations for data import and synchronization.
#### OAuth2 Connector Configuration
```yaml
connectors:
- name: "service-name"
type: "oauth2"
config:
client-id: "oauth2-client-id"
client-secret: "oauth2-client-secret"
redirect-uri: "https://your-domain/api/console/v1/connectors/complete"
auth-url: "https://service.com/oauth/authorize"
token-url: "https://service.com/oauth/token"
scopes:
- "scope1"
- "scope2"
```
##### `connectors[].name` (string)
**Required**
Unique identifier for the connector instance.
##### `connectors[].type` (string)
**Required**
Connector protocol type. Currently supported: `"oauth2"`.
##### `connectors[].config.client-id` (string)
**Required**
OAuth2 client identifier provided by the external service.
##### `connectors[].config.client-secret` (string)
**Required**
OAuth2 client secret provided by the external service.
##### `connectors[].config.redirect-uri` (string)
**Required**
OAuth2 redirect URI. Must match the URI registered with the external service.
##### `connectors[].config.auth-url` (string)
**Required**
OAuth2 authorization endpoint URL.
##### `connectors[].config.token-url` (string)
**Required**
OAuth2 token exchange endpoint URL.
##### `connectors[].config.scopes` (array of strings)
**Optional**
List of OAuth2 scopes to request during authorization.
## Security Considerations
### Secrets Management
- **Encryption Keys**: The `encryption-key` should be generated using a cryptographically secure random number generator and stored securely.
- **Database Credentials**: Use strong passwords and consider database connection encryption for production deployments.
- **Cookie Secrets**: Authentication and trust auth secrets should be unique, random, and at least 32 bytes long.
- **API Keys**: Store external service API keys securely and rotate them regularly.
### Network Security
- Configure appropriate firewall rules to restrict access to the Probod service.
- Use TLS/SSL termination at the load balancer or reverse proxy level.
- Implement proper CORS configuration to prevent unauthorized cross-origin requests.
### Database Security
- Use TLS encryption for database connections in production.
- Implement database user permissions following the principle of least privilege.
- Regular database backups and security updates are recommended.
## Environment-Specific Configuration
### Development
For development environments, you can use the provided example configurations in the `cfg/` directory:
- `cfg/dev.yaml` - Basic development configuration
- `cfg/gearnode.yaml` - Advanced development configuration with connectors
### Production
Production deployments should:
1. Use strong, unique secrets for all authentication components
2. Enable TLS for all external communications
3. Use managed database services with encryption at rest
4. Implement proper monitoring and logging
5. Regular security updates and vulnerability assessments
## Troubleshooting
### Common Configuration Issues
1. **Database Connection Failures**: Verify database credentials, network connectivity, and certificate configuration.
2. **Authentication Problems**: Check cookie domain settings and secret key configuration.
3. **External Connector Issues**: Verify OAuth2 client credentials and redirect URI configuration.
4. **File Upload Problems**: Ensure AWS credentials and S3 bucket configuration are correct.
### Logging
Probod provides structured logging that can help diagnose configuration issues. Enable debug logging by setting appropriate log levels in your deployment environment.
## Configuration Validation
Probod validates configuration on startup and will report specific errors for:
- Missing required fields
- Invalid data formats
- Unreachable external services
- Invalid secrets or keys
Review startup logs carefully to identify and resolve configuration issues.

311
docs/INSTALLATION.md Normal file
View File

@@ -0,0 +1,311 @@
# Probo Installation Guide
This document provides comprehensive installation instructions for the Probo compliance management platform daemon (`probod`).
## Installation Methods
Probo can be deployed using either the official Docker image or pre-compiled binaries available through GitHub releases.
## Docker Installation
The official Docker images are available on GitHub Container Registry and support multiple architectures:
- **Multi-architecture image**: `ghcr.io/getprobo/probo:latest`
- **AMD64 (x86_64)**: `ghcr.io/getprobo/probo:latest-amd64`
- **ARM64**: `ghcr.io/getprobo/probo:latest-arm64`
### Basic Docker Setup
To run Probo using Docker:
```bash
docker run -d \
--name probod \
-p 8080:8080 \
-v /path/to/config.yaml:/etc/probod/config.yaml \
ghcr.io/getprobo/probo:latest
```
### Docker Compose Setup
For a complete setup with dependencies, you can use Docker Compose:
```yaml
version: "3.8"
services:
probod:
image: ghcr.io/getprobo/probo:latest
ports:
- "8080:8080"
volumes:
- ./config.yaml:/etc/probod/config.yaml
- ./data:/data
environment:
- PROBOD_CONFIG=/etc/probod/config.yaml
depends_on:
- postgres
- minio
postgres:
image: postgres:15
environment:
POSTGRES_DB: probod
POSTGRES_USER: probod
POSTGRES_PASSWORD: probod
volumes:
- postgres_data:/var/lib/postgresql/data
ports:
- "5432:5432"
minio:
image: minio/minio:latest
command: server /data --console-address ":9001"
environment:
MINIO_ROOT_USER: probod
MINIO_ROOT_PASSWORD: thisisnotasecret
volumes:
- minio_data:/data
ports:
- "9000:9000"
- "9001:9001"
volumes:
postgres_data:
minio_data:
```
### Docker Architecture Support
The Docker images support the following architectures:
- **linux/amd64** (x86_64) - Standard 64-bit Intel/AMD processors
- **linux/arm64** - ARM 64-bit processors (Apple Silicon, AWS Graviton, etc.)
Multi-architecture images automatically select the appropriate variant for your platform.
## Binary Installation
Pre-compiled binaries are available for download from the [GitHub releases page](https://github.com/getprobo/probo/releases).
### Supported Platforms
The following platforms are officially supported:
- **Windows**: AMD64 (x86_64)
- **macOS**: AMD64 (x86_64) and ARM64 (Apple Silicon)
- **Linux**: AMD64 (x86_64) and ARM64 (via Docker)
If your specific platform is not available, please contact us, and we may be able to add support for additional architectures.
### Installation Steps
1. **Download the Binary**
Visit the [GitHub releases page](https://github.com/getprobo/probo/releases) and download the appropriate archive for your platform:
- **Windows**: `probod_Windows_x86_64.zip`
- **macOS (Intel)**: `probod_Darwin_x86_64.tar.gz`
- **macOS (Apple Silicon)**: `probod_Darwin_arm64.tar.gz`
2. **Extract the Archive**
```bash
# For tar.gz files (macOS/Linux)
tar -xzf probod_Darwin_x86_64.tar.gz
# For zip files (Windows)
# Use your preferred extraction tool
```
3. **Install the Binary**
**macOS/Linux:**
```bash
# Move to a directory in your PATH
sudo mv probod /usr/local/bin/
# Make executable (if not already)
sudo chmod +x /usr/local/bin/probod
```
**Windows:**
```cmd
# Move probod.exe to a directory in your PATH
# Or add the current directory to your PATH environment variable
```
4. **Verify Installation**
```bash
probod --version
```
### Running the Binary
Once installed, you can run Probo with a configuration file:
```bash
# Using the default configuration location
probod --config /etc/probod/config.yaml
# Or specify a custom configuration file
probod --config ./my-config.yaml
```
## System Requirements
### Minimum Requirements
- **CPU**: 1 core, 2 GHz
- **Memory**: 1 GB RAM
- **Storage**: 10 GB available space
- **Network**: Internet connectivity for external integrations
### Recommended Requirements
- **CPU**: 2+ cores, 2.4 GHz
- **Memory**: 4 GB RAM
- **Storage**: 50 GB available space (SSD preferred)
- **Network**: Stable internet connection
### Dependencies
Probo requires the following external services:
1. **PostgreSQL Database** (version 12 or higher)
2. **S3-Compatible Storage** (AWS S3, MinIO, etc.)
3. **Chrome/Chromium** (for PDF generation via Chrome DevTools Protocol)
Optional dependencies:
- **SMTP Server** (for email notifications)
- **OpenAI API** (for AI-powered features)
## Quick Start
### 1. Database Setup
Create a PostgreSQL database for Probo:
```sql
CREATE DATABASE probod;
CREATE USER probod WITH PASSWORD 'your_secure_password';
GRANT ALL PRIVILEGES ON DATABASE probod TO probod;
```
### 2. Configuration
Create a basic configuration file (`config.yaml`):
```yaml
probod:
hostname: "localhost:8080"
encryption-key: "your-base64-encoded-encryption-key"
pg:
addr: "localhost:5432"
username: "probod"
password: "your_secure_password"
database: "probod"
aws:
region: "us-east-1"
bucket: "probod"
access-key-id: "your-access-key"
secret-access-key: "your-secret-key"
endpoint: "http://localhost:9000" # For MinIO
```
### 3. Start the Service
```bash
# Using Docker
docker run -d \
--name probod \
-p 8080:8080 \
-v ./config.yaml:/etc/probod/config.yaml \
ghcr.io/getprobo/probo:latest
# Using Binary
probod --config config.yaml
```
### 4. Access the Application
Open your web browser and navigate to `http://localhost:8080` to access the Probo web interface.
## Production Deployment
### Security Considerations
1. **Use strong, unique secrets** for all authentication components
2. **Enable TLS** for all external communications
3. **Use managed database services** with encryption at rest
4. **Implement proper monitoring** and logging
5. **Regular security updates** and vulnerability assessments
### Load Balancing
For high-availability deployments, consider using:
- **Reverse Proxy**: Nginx, HAProxy, or cloud load balancers
- **Database Clustering**: PostgreSQL with read replicas
- **File Storage**: Distributed S3-compatible storage
### Monitoring
Probo provides metrics and health checks:
- **Health Check**: `GET /health`
- **Metrics**: Prometheus-compatible metrics endpoint
- **Logging**: Structured JSON logging with configurable levels
## Troubleshooting
### Common Installation Issues
1. **Permission Denied (Binary)**
```bash
chmod +x probod
```
2. **Database Connection Failed**
- Verify database credentials and network connectivity
- Check PostgreSQL is running and accepting connections
3. **Docker Image Pull Failed**
```bash
docker login ghcr.io
docker pull ghcr.io/getprobo/probo:latest
```
4. **Port Already in Use**
```bash
# Find process using port 8080
lsof -i :8080
# Or use a different port in configuration
```
### Getting Help
- **Documentation**: Check the [configuration reference](./CONFIGURATION.md)
- **GitHub Issues**: Report bugs and request features
- **Community**: Join our community discussions
### Log Analysis
Enable debug logging to troubleshoot issues:
```yaml
probod:
# ... other configuration
log-level: debug
```
Check startup logs for configuration validation errors and service initialization issues.

137
docs/README.md Normal file
View File

@@ -0,0 +1,137 @@
# Probod Operations Documentation
This documentation provides operational guidance for deploying, configuring, and managing the Probod compliance management daemon.
## Service Components
Probod requires the following components for operation:
- **probod**: Main service daemon
- **PostgreSQL**: Database backend (version 17+)
- **S3-Compatible Storage**: Object storage (AWS S3, MinIO, etc.)
- **Chrome DevTools Protocol**: PDF generation service
## Deployment Guide
- **[Installation Guide](./INSTALLATION.md)** - Docker and binary deployment procedures
- **[Configuration Reference](./CONFIGURATION.md)** - Complete configuration options and examples
## Quick Deployment
### Docker Deployment
```bash
docker run -d \
--name probod \
-p 8080:8080 \
-v /path/to/config.yaml:/etc/probod/config.yaml \
ghcr.io/getprobo/probo:latest
```
### Service Verification
```bash
# Health check
curl http://localhost:8080/health
# Service logs
docker logs probod
```
## System Requirements
### Minimum Requirements
- **CPU**: 1 core, 2 GHz
- **Memory**: 1 GB RAM
- **Storage**: 10 GB available space
- **Database**: PostgreSQL 12+
- **Storage Backend**: S3-compatible object storage
### Recommended for Production
- **CPU**: 2+ cores, 2.4 GHz
- **Memory**: 4 GB RAM
- **Storage**: 50 GB SSD
- **Database**: Managed PostgreSQL with backups
- **Load Balancer**: For high availability deployments
## External Dependencies
### Required Services
- **PostgreSQL Database**: Primary data storage
- **S3-Compatible Storage**: File and document storage (AWS S3, MinIO, etc.)
- **Chrome/Chromium**: PDF generation via DevTools Protocol
### Optional Integrations
- **SMTP Server**: Email notifications
- **OpenAI API**: AI-powered features
- **OAuth2 Providers**: External service integrations (GitHub, Slack, etc.)
## Production Deployment
### High Availability
- Deploy multiple probod instances behind a load balancer
- Use managed PostgreSQL with read replicas
- Configure distributed object storage
- Implement monitoring and alerting
### Security Configuration
- Enable TLS for all external communications
- Use strong, unique secrets for authentication components
- Configure network security groups and firewalls
- Regular security updates and vulnerability scanning
## Operations
### Monitoring
- **Health Check**: `GET /health`
- **Metrics**: Prometheus-compatible metrics at `/metrics` endpoint
- **Tracing**: OpenTelemetry distributed tracing support
- **Logging**: Structured JSON logging with configurable levels
#### Configuration-Based Observability
Probod provides built-in observability configured through the main configuration file:
- **Prometheus Metrics**: Configure `unit.metrics.addr` for metrics endpoint
- **OpenTelemetry Tracing**: Configure `unit.tracing` section for trace collection
- **Structured Logging**: Automatic JSON logging with correlation IDs
#### Configuration Example
```yaml
unit:
metrics:
addr: "0.0.0.0:8081"
tracing:
addr: "tempo:4317"
```
See [Configuration Reference](./CONFIGURATION.md#telemetry-and-observability) for complete details.
### Backup and Recovery
- **Database**: Regular PostgreSQL backups with point-in-time recovery
- **Object Storage**: S3 versioning and cross-region replication
- **Configuration**: Version control all configuration files
### Troubleshooting
Common operational issues:
- Verify database connectivity and credentials
- Check S3 storage access permissions
- Validate configuration file syntax
- Review service logs for startup errors
## Support
- **Repository**: [getprobo/probo](https://github.com/getprobo/probo)
- **Issues**: Report operational problems on GitHub
- **Releases**: [GitHub releases](https://github.com/getprobo/probo/releases)