IBM Cloud specific tools and diagnostic commands for troubleshooting performance

Accessing monitoring dashboards

1. Navigate to your MongoDB deployment in IBM Cloud console.
2. Click Monitoring in the left navigation.
3. Click Launch Monitoring to open the Sysdig dashboard.

Key metrics to track:

• Platform metrics:

  • CPU utilization - target: < 75% sustained
  • Disk utilization - target: < 80%
  • Disk IOPS - monitor for saturation
  • Network throughput - identify bandwidth constraints

• MongoDB-specific metrics:

  • Operations per second - track workload patterns
  • Active connections - monitor against plan limits
  • Replication lag - target: < 1 second
  • Query execution time - identify slow queries
  • Cache hit ratio - target: > 95%

Setting up alerts

Create alerts for critical thresholds:

Alert: High CPU Usage
Condition: CPU > 80% for 10 minutes
Action: Notify operations team

Alert: Replication Lag
Condition: Replication lag > 5 seconds
Action: Page on-call engineer

Alert: Disk Space
Condition: Disk usage > 85%
Action: Trigger scaling workflow

Creating custom dashboards

1. In Sysdig, click Dashboards > Create Dashboard.
2. Add panels for key metrics.
3. Use filters to focus on your MongoDB deployment.
4. Save and share with your team.

Example dashboard layout

• Row 1: CPU, memory, disk utilization
• Row 2: Operations per second, active connections
• Row 3: Replication lag, query performance
• Row 4: Cache statistics, lock contention

Historical analysis

• Use the time range selector for historical data
• Compare current metrics with baseline
• Identify trends and patterns
• Correlate events with performance changes

Recommended actions:

• Set up alerts before issues occur.
• Review dashboards daily.
• Establish baseline metrics.
• Document normal patterns compared to abnormal patterns.
• Use metrics for capacity planning.

Accessing Activity Tracker

1. Navigate to Observability > Activity Tracker in the IBM Cloud console.
2. Select your region.
3. Filter events by your MongoDB instance.

Key events to monitor:

• Configuration changes:

  • Scaling operations (CPU, memory, disk)
  • Backup configuration changes
  • Network configuration updates
  • User access modifications

• Performance-impacting events

  • Database restarts
  • Failover events
  • Maintenance operations
  • Index creation and deletion

Correlating events with performance issues

1. Note the timestamp of performance degradation.
2. Search Activity Tracker for events around that time.
3. Look for configuration changes or administrative actions.
4. Correlate with monitoring metrics.

Example event analysis

Event: Database scaled from 2GB to 4GB RAM
Time: 2024-01-15 14:30:00 UTC
Impact: Temporary connection disruption (30 seconds)
Result: Improved performance after scaling

Audit trail for compliance

• Track who made changes and when
• Maintain compliance with security policies
• Review access patterns
• Identify unauthorized changes

Recommended actions:

• Review Activity Tracker logs regularly.
• Set up alerts for critical events.
• Document change management procedures.
• Correlate events with performance metrics.
• Use for post-incident analysis.

Vertical scaling (compute and memory)

Scale CPU and memory resources to handle increased workload.

• Using the IBM Cloud console:

  1. Navigate to your MongoDB deployment.
  2. Click Resources in the left navigation.
  3. Adjust Memory and CPU sliders.
  4. Review cost impact.
  5. Click Scale.

• Using the IBM Cloud CLI:

  # Scale memory to 8GB and CPU to 4 cores
  ibmcloud cdb deployment-groups-set <deployment-id> member \
    --memory 8192 \
    --cpu-allocation 4
  

Considerations

• Brief connection disruption during scaling.
• Plan for 5-10 minutes downtime.
• Scale proactively before saturation.
• Monitor metrics after scaling.

Horizontal scaling (replica set members)

Add replica set members for read scaling and high availability.

• Using the IBM Cloud console:

  1. Navigate to Resources.
  2. Adjust Members slider.
  3. Review configuration.
  4. Click Scale.

• Using the IBM Cloud CLI:

  # Add a replica set member
  ibmcloud cdb deployment-groups-set <deployment-id> member \
    --members 4
  

Benefits

• Distribute read load across secondaries
• Improved fault tolerance
• Better geographic distribution
• No downtime for adding members

Storage scaling

Increase disk space and IOPS for better performance.

• Using the IBM Cloud console:

  1. Navigate to Resources.
  2. Adjust Disk slider.
  3. Review IOPS allocation.
  4. Click Scale.

• Using the IBM Cloud CLI:

  # Scale disk to 100GB
  ibmcloud cdb deployment-groups-set <deployment-id> member \
    --disk-allocation 102400
  

Important information:

• Storage can only be increased, not decreased
• IOPS scale with disk size
• No downtime for storage scaling
• Monitor disk usage trends

Scaling best practices

| Disk latency | Increase disk size for more IOPS | | Connection limits | Scale to higher tier | | Read-heavy workload | Add replica members | | Write-heavy workload | Scale CPU and memory |

Cost optimization

• Right size your deployment.
• Use monitoring to identify actual needs.
• Scale down during low-traffic periods (if supported).
• Consider reserved capacity for predictable workloads.

Automation

# Example: Auto-scale based on CPU threshold
if [ $(ibmcloud cdb deployment-metrics <deployment-id> --metric cpu) -gt 80 ]; then
  ibmcloud cdb deployment-groups-set <deployment-id> member --cpu-allocation 6
fi

Installing IBM Cloud CLI

# Install IBM Cloud CLI
curl -fsSL https://clis.cloud.ibm.com/install/linux | sh

# Install databases plugin
ibmcloud plugin install cloud-databases

Essential diagnostic commands

• Get deployment information:

  # List all MongoDB deployments
  ibmcloud cdb deployments --type mongodb
  
  # Get specific deployment details
  ibmcloud cdb deployment <deployment-id>
  

• Check deployment status:

  # Get deployment status
  ibmcloud cdb deployment-status <deployment-id>
  
  # Get connection strings
  ibmcloud cdb deployment-connections <deployment-id>
  

• Monitor metrics:

  # Get CPU metrics
  ibmcloud cdb deployment-metrics <deployment-id> --metric cpu
  
  # Get memory metrics
  ibmcloud cdb deployment-metrics <deployment-id> --metric memory
  
  # Get disk metrics
  ibmcloud cdb deployment-metrics <deployment-id> --metric disk
  

• Scaling operations:

  # Scale memory
  ibmcloud cdb deployment-groups-set <deployment-id> member \
    --memory 16384
  
  # Scale CPU
  ibmcloud cdb deployment-groups-set <deployment-id> member \
    --cpu-allocation 8
  
  # Scale disk
  ibmcloud cdb deployment-groups-set <deployment-id> member \
    --disk-allocation 204800
  

• Backup operations:

  # List backups
  ibmcloud cdb backups <deployment-id>
  
  # Get backup information
  ibmcloud cdb backup <backup-id>
  

Using the IBM Cloud API

• Authentication:

  # Get IAM token
  export IAM_TOKEN=$(ibmcloud iam oauth-tokens --output json | jq -r '.iam_token')
  

• Get deployment metrics using the API:

  # Get metrics
  curl -X GET \
    "https://api.{region}.databases.cloud.ibm.com/v5/deployments/{deployment-id}/metrics" \
    -H "Authorization: ${IAM_TOKEN}"
  

• Scale deployment using the API:

  # Scale resources
  curl -X PATCH \
    "https://api.{region}.databases.cloud.ibm.com/v5/deployments/{deployment-id}/groups/member" \
    -H "Authorization: ${IAM_TOKEN}" \
    -H "Content-Type: application/json" \
    -d '{
      "memory": {
        "allocation_mb": 16384
      },
      "cpu": {
        "allocation_count": 8
      }
    }'
  

• Sample diagnostic script:

  #!/bin/bash
  # MongoDB Performance Check Script
  
  DEPLOYMENT_ID="your-deployment-id"
  
  echo "=== MongoDB Performance Diagnostics ==="
  echo ""
  
  # Check CPU
  CPU=$(ibmcloud cdb deployment-metrics $DEPLOYMENT_ID --metric cpu --output json | jq -r '.metrics[0].value')
  echo "CPU Usage: ${CPU}%"
  if [ $(echo "$CPU > 80" | bc) -eq 1 ]; then
    echo "⚠️  WARNING: High CPU usage detected"
  fi
  
  # Check Memory
  MEMORY=$(ibmcloud cdb deployment-metrics $DEPLOYMENT_ID --metric memory --output json | jq -r '.metrics[0].value')
  echo "Memory Usage: ${MEMORY}%"
  if [ $(echo "$MEMORY > 80" | bc) -eq 1 ]; then
    echo "⚠️  WARNING: High memory usage detected"
  fi
  
  # Check Disk
  DISK=$(ibmcloud cdb deployment-metrics $DEPLOYMENT_ID --metric disk --output json | jq -r '.metrics[0].value')
  echo "Disk Usage: ${DISK}%"
  if [ $(echo "$DISK > 80" | bc) -eq 1 ]; then
    echo "⚠️  WARNING: High disk usage detected"
  fi
  
  # Check Status
  STATUS=$(ibmcloud cdb deployment-status $DEPLOYMENT_ID --output json | jq -r '.status')
  echo "Deployment Status: ${STATUS}"
  
  echo ""
  echo "=== Diagnostics Complete ==="
  

• Automation recommendations

  • Schedule regular health checks.
  • Integrate with monitoring systems.
  • Automate scaling based on thresholds.
  • Create alerts for critical metrics.
  • Log all operations for audit trail.

Private endpoints (recommended)

Benefits:

• Lower latency
• Enhanced security
• No internet egress charges
• Better performance for IBM Cloud workloads

Setup:

1. Navigate to Settings > Endpoints.
2. Enable Private endpoint.
3. Update connection strings in applications.

Connection string example:

mongodb://user:pass@host.private.databases.appdomain.cloud:port/database?authSource=admin&replicaSet=replset

Public endpoints

Use cases:

• External applications
• Development and testing
• Hybrid cloud scenarios

Security considerations:

• Use IP allowlisting.
• Enforce TLS/SSL.
• Rotate credentials regularly.

Service endpoints

IBM Cloud service endpoints provide optimized connectivity within IBM Cloud.

Benefits

• Reduced latency
• No public internet traversal
• Improved security posture
• Cost savings on bandwidth

Configuration

# Enable service endpoint
ibmcloud cdb deployment-service-endpoint-enable <deployment-id>

Multi-zone deployment considerations

Databases for MongoDB can span multiple availability zones.

Best practices

• Deploy applications in the same region.
• Use read preferences to minimize latency.
• Consider nearest read preference for multi-zone apps.
• Monitor replication lag between zones.

Network latency troubleshooting

• Measure latency from application

  # Test connection latency
  time mongo "mongodb://host:port/database" --eval "db.runCommand({ping: 1})"
  

• Check from IBM Cloud shell

  # Ping test (if ICMP allowed)
  ping -c 10 your-mongodb-host.databases.appdomain.cloud
  
  # TCP connection test
  nc -zv your-mongodb-host.databases.appdomain.cloud 27017
  

MongoDB connection diagnostics

// Check network latency
db.runCommand({ ping: 1 })

// Check connection pool stats
db.serverStatus().connections

Geographic distribution

For globally distributed applications:

Strategies

• Single region: Lowest latency, single point of failure
• Multi-region with read replicas: Read scaling, eventual consistency
• Cross-region replication: Disaster recovery, higher latency

Recommendations

• Place database close to primary user base.
• Use CDN for static content.
• Implement application-level caching.
• Consider data residency requirements.

Bandwidth optimization

• Use projections to limit data transfer.
• Implement pagination for large result sets.
• Compress data at application level.
• Use bulk operations to reduce round trips.

Connection pooling best practices

// Node.js example
const client = new MongoClient(uri, {
  maxPoolSize: 50,
  minPoolSize: 10,
  maxIdleTimeMS: 30000,
  serverSelectionTimeoutMS: 5000,
  socketTimeoutMS: 45000
});