play.hpccsystems.com

This repository contains scripts and configuration for managing the HPCC Platform on play.hpccsystems.com, including an automated version management system.

~/bin

hpcc_fetch_and_upgrade

• Comprehensive HPCC Platform update automation
• Fetches latest releases from GitHub API and filters out release candidates
• Compares versions to identify newest available release
• Downloads newer versions automatically using hpcc_fetch_enhanced
• Manages file retention (keeps minimum of 3 release files)
• Executes hpcc_upgrade_enhanced for system updates
• Features:
  • ✅ Automatic version detection from GitHub
  • ✅ Gold release filtering (excludes rc1, rc2, etc.)
  • ✅ Semantic version comparison with revision support
  • ✅ Smart file management with retention policy
  • ✅ Comprehensive logging with categorized output
  • ✅ Robust error handling and graceful failures
• Configuration:
  • Release Directory: /home/innovate/
  • File Pattern: hpccsystems-platform-community_*noble_amd64_withsymbols.deb
  • Repository: hpcc-systems/HPCC-Platform
• Usage: ./hpcc_fetch_and_upgrade

apt_update_all.sh

• Bring the OS and all installed .deb packages up to date

azure_make_mounted.sh

• Mount an external Azure disk permanently

hpcc_config

• Launch HPCC's configmgr

hpcc_fetch <version>

• Download the Ubuntu 24.04 version of the platform
• Will append '-1' to the version if needed

hpcc_fetch_enhanced <version>

• Enhanced version of hpcc_fetch with robust error handling
• Download the Ubuntu Noble version of the platform
• Comprehensive input validation and file verification
• Automatic retry logic and timeout handling
• File size validation and cleanup on failure
• Skip download if file already exists

hpcc_snapshot <create | restore | wipe>

• Manage archived copies of /lib/HPCCSystems

hpcc_start

• Start the installed platform

hpcc_stop

• Stop the installed platform

hpcc_upgrade <package_path>

• Upgrades the current platform, installing the given package

hpcc_upgrade_enhanced <package_path>

• Enhanced version of hpcc_upgrade with comprehensive error handling
• Validates package file existence, type (.deb), and size
• Step-by-step error checking for stop/install/start operations
• Graceful recovery if installation fails
• Service verification after upgrade completion
• Detailed progress reporting and error messages

my_ip.sh

• What is my IP?

renew_lets_encrypt_cert.sh

• Renews Let's Encrypt certificate

search

• Shortcut for the find command line utility

wipe_hpcc.sh

• Restarts the cluster, wiping all data in the process

---

/etc

• cron.d
  • certbot
• cron.hourly
  • hpcc_certificates
• cron.daily
  • hpcc_logs
  • logrotate

---

Notes

• Platform configuration disallows embedded (non-ECL) code execution
• Cluster is wiped daily to mitigate problematic uploads

---

Azure

• Subscription: us-hpccsystems-dev
• Resource Group: play-vm-rg
• VM: play-vm-hpcc
  • IP: 20.163.232.157
  • OS: Ubuntu 24.04

---

Periodic Activities

Automated Updates The hpcc_fetch_and_upgrade script can be scheduled to run automatically for hands-off HPCC Platform updates.

Current Manual Updates - Every Friday morning [Can choose any day. Preferably early morning to avoid interruptions]

• Bring OS and packages up to date:
  • apt_update_all.sh
• Install latest HPCC Platform:
  • hpcc_fetch <version>
  • hpcc_upgrade <package_file>