Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 77 additions & 0 deletions .github/instructions/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# Instructions Summary

This directory contains comprehensive instructions for AI assistants working with the HPCC-JS-WASM repository.

## File Overview

### Main Instructions
- **[main.md](main.md)** - Start here! Repository overview, architecture, and basic guidance
- **[workflow.md](workflow.md)** - Step-by-step development workflows for common tasks
- **[patterns.md](patterns.md)** - Quick reference for package patterns and usage examples
- **[troubleshooting.md](troubleshooting.md)** - Debugging guide for common issues

### Package-Specific Instructions
- **[../../packages/graphviz/.copilot-package.md](../../packages/graphviz/.copilot-package.md)** - Graphviz package specifics
- **[../../packages/duckdb/.copilot-package.md](../../packages/duckdb/.copilot-package.md)** - DuckDB package specifics

## Quick Start for AI Assistants

1. **First Time Setup**: Read [main.md](main.md) for repository overview
2. **Development Tasks**: Follow workflows in [workflow.md](workflow.md)
3. **Package Usage**: Check patterns in [patterns.md](patterns.md)
4. **Problems?**: Consult [troubleshooting.md](troubleshooting.md)

## Key Points for AI Assistants

- **Monorepo Structure**: 8 packages, each providing WASM versions of C++ libraries
- **Build Complexity**: Requires C++ compilation for full functionality
- **TypeScript First**: Many operations can be done with TypeScript-only builds
- **Test Expectations**: Fresh clone tests will fail without WASM builds (expected)
- **Safe Operations**: Documentation changes, TypeScript modifications don't need full build

## Most Common Scenarios

### Making TypeScript Changes
```bash
npm ci
npm run lint
npm run build-ws # May fail without WASM, but can still work on TS
```

### Full Development Setup
```bash
npm ci
npm run install-build-deps # Requires system tools
npm run build-cpp # Compiles C++ to WASM
npm run build-ws # Builds TypeScript
npm run test # Full test suite
```

### Quick Fix Workflow
```bash
# Make changes
npm run lint
npm run lint-fix # Auto-fix linting issues
# Test specific package if needed
cd packages/[package] && npm test
```

## When to Escalate

- C++ compilation errors requiring emscripten expertise
- System dependency issues (cmake, python, etc.)
- Platform-specific problems (Windows, specific Linux distributions)
- Memory/performance issues beyond normal optimization

## Success Metrics

You've successfully used these instructions if:
- You understand the repository structure and build system
- You can make TypeScript changes safely
- You can diagnose and resolve common build/test failures
- You know when to attempt full builds vs. TypeScript-only changes
- You can help users with package-specific questions

---

*These instructions are designed to help AI assistants provide better support for the HPCC-JS-WASM repository. They should be updated as the repository evolves.*
223 changes: 223 additions & 0 deletions .github/instructions/main.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,223 @@
# Copilot Instructions for HPCC-JS-WASM

This document provides guidance for AI assistants working with the HPCC-JS-WASM repository.

## Repository Overview

This is a **monorepo** that provides WebAssembly (WASM) versions of popular C++ libraries for use in Node.js, web browsers, and JavaScript applications. It uses **Lerna** for monorepo management and **npm workspaces**.

### Packages Provided

1. **@hpcc-js/wasm-base91** - Base91 encoding/decoding library
2. **@hpcc-js/wasm-duckdb** - DuckDB embedded database
3. **@hpcc-js/wasm-expat** - Expat XML parser
4. **@hpcc-js/wasm-graphviz** - Graphviz graph visualization library
5. **@hpcc-js/wasm-graphviz-cli** - Command-line interface for Graphviz
6. **@hpcc-js/wasm-llama** - Llama.cpp AI model library
7. **@hpcc-js/wasm-zstd** - Zstandard compression library
8. **@hpcc-js/wasm** - Meta package for backward compatibility

## Architecture

### File Structure
```
β”œβ”€β”€ packages/ # Individual WASM packages
β”‚ β”œβ”€β”€ base91/ # Base91 package
β”‚ β”œβ”€β”€ duckdb/ # DuckDB package
β”‚ β”œβ”€β”€ expat/ # Expat package
β”‚ β”œβ”€β”€ graphviz/ # Graphviz package
β”‚ β”œβ”€β”€ graphviz-cli/ # Graphviz CLI package
β”‚ β”œβ”€β”€ llama/ # Llama package
β”‚ β”œβ”€β”€ wasm/ # Meta package
β”‚ └── zstd/ # Zstd package
β”œβ”€β”€ src-cpp/ # Shared C++ source code
β”œβ”€β”€ scripts/ # Build and utility scripts
β”œβ”€β”€ docs/ # Documentation (VitePress)
└── .vscode/ # VSCode configuration
```

### Package Structure Pattern
Each package typically contains:
```
packages/[name]/
β”œβ”€β”€ src/ # TypeScript source
β”œβ”€β”€ src-cpp/ # C++ source code
β”œβ”€β”€ tests/ # Test files
β”œβ”€β”€ package.json # Package configuration
└── vitest.config.ts # Test configuration
```

## Development Workflow

### Prerequisites for Full Development
```bash
# Install dependencies
npm ci

# Install build dependencies (requires system tools)
npm run install-build-deps # Installs emsdk, vcpkg, playwright

# Build C++ to WASM (requires emscripten)
npm run build-cpp

# Build TypeScript packages
npm run build-ws
```

### Quick Development (TypeScript only)
```bash
# Install dependencies
npm ci

# Build only TypeScript (without C++ compilation)
npm run build-ws

# Run linting
npm run lint

# Note: Tests may fail without WASM builds
```

### Testing
```bash
# Run all tests (requires full build)
npm run test

# Run specific package tests
cd packages/graphviz && npm test

# Run browser tests
npm run test-browser

# Run node tests
npm run test-node
```

## Common Patterns

### WASM Library Loading Pattern
All WASM libraries follow this pattern:
```typescript
import { LibraryName } from "@hpcc-js/wasm-libraryname";

// Async loading required
const library = await LibraryName.load();

// Use library methods
const result = library.someMethod(input);
```

### Package Export Structure
```typescript
// src/index.ts - Main entry point
export * from "./libraryname.ts";

// src/libraryname.ts - Main implementation
export class LibraryName {
static async load(): Promise<LibraryName> { ... }
someMethod(input: string): string { ... }
version(): string { ... }
}
```

### Testing Patterns
```typescript
import { describe, it, expect } from "vitest";
import { LibraryName } from "@hpcc-js/wasm-libraryname";

describe("LibraryName", () => {
it("basic functionality", async () => {
const lib = await LibraryName.load();
const result = lib.someMethod("test");
expect(result).toBeDefined();
});
});
```

## Key Technologies

- **TypeScript** - Primary language for package implementations
- **C++** - Source libraries compiled to WASM
- **Emscripten** - C++ to WASM compilation toolchain
- **Lerna** - Monorepo management
- **Vitest** - Testing framework (with browser support)
- **ESBuild** - TypeScript bundling
- **VitePress** - Documentation generation

## Build System

### C++ Compilation Chain
1. **vcpkg** - C++ package manager for dependencies
2. **emscripten** - Compiles C++ to WASM
3. **IDL files** - WebIDL bindings for C++ classes
4. **Custom esbuild plugins** - Handle WASM imports

### TypeScript Build
1. **tsc** - Type generation
2. **esbuild** - Bundling (ESM, CJS, UMD formats)
3. **Custom plugins** - WASM asset handling

## Common Issues and Solutions

### Build Failures
- **Missing WASM files**: Run `npm run build-cpp` first
- **Type errors**: Ensure all packages are built with `npm run build-ws`
- **Test failures in fresh clone**: Expected - requires full build process

### Package Resolution Issues
- Packages depend on each other being built
- Use relative imports for local development
- Run `lerna run build` to build all packages

### WASM Loading Issues
- WASM files must be accessible at runtime
- Browser requires proper MIME types for .wasm files
- Node.js needs appropriate file system access

## Debugging

### VSCode Configuration
- Launch configurations provided for browser and Node.js debugging
- Tasks configured for watch mode development
- CMake integration for C++ development

### Log Levels
Most libraries support debug output:
```typescript
// Enable debug logging
const lib = await LibraryName.load({ debug: true });
```

## Documentation

- **API Docs**: Generated with TypeDoc at https://hpcc-systems.github.io/hpcc-js-wasm/
- **Examples**: See `/examples` directory and HTML files in root
- **Package READMEs**: Each package has specific documentation

## AI Assistant Guidelines

### When Making Changes
1. **Understand the monorepo structure** - changes may affect multiple packages
2. **Follow existing patterns** - each package follows similar structure
3. **Test incrementally** - build and test after each significant change
4. **Consider WASM implications** - changes to C++ require full rebuild
5. **Update documentation** - maintain TypeDoc comments and READMEs

### Safe Operations (No Full Build Required)
- TypeScript code changes in `src/` directories
- Test file modifications
- Documentation updates
- Package.json script modifications

### Operations Requiring Full Build
- C++ source code changes in `src-cpp/` directories
- WebIDL (.idl) file changes
- CMakeLists.txt modifications
- New WASM library additions

### Before Submitting Changes
1. Run `npm run lint` to check code style
2. Run `npm run build-ws` to ensure TypeScript builds
3. If C++ was modified, run full build cycle
4. Test affected packages individually
5. Update relevant documentation
Loading
Loading