From 6dcd9829a66ebc8134b1e3d3e20bc2cf56532f21 Mon Sep 17 00:00:00 2001
From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com>
Date: Fri, 11 Jul 2025 06:08:57 +0000
Subject: [PATCH 1/4] Initial plan


From 6aa9bb2716215691edc7e8ba0637fabb133a8191 Mon Sep 17 00:00:00 2001
From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com>
Date: Fri, 11 Jul 2025 06:20:34 +0000
Subject: [PATCH 2/4] Add comprehensive copilot instructions for AI assistants

Co-authored-by: GordonSmith <966863+GordonSmith@users.noreply.github.com>
---
 .copilot-instructions.md              | 223 ++++++++++++++
 .copilot-patterns.md                  | 259 ++++++++++++++++
 .copilot-troubleshooting.md           | 425 ++++++++++++++++++++++++++
 .copilot-workflow.md                  | 373 ++++++++++++++++++++++
 .vscode/settings.json                 |  37 ++-
 README.md                             |  28 +-
 packages/duckdb/.copilot-package.md   | 398 ++++++++++++++++++++++++
 packages/graphviz/.copilot-package.md | 245 +++++++++++++++
 8 files changed, 1978 insertions(+), 10 deletions(-)
 create mode 100644 .copilot-instructions.md
 create mode 100644 .copilot-patterns.md
 create mode 100644 .copilot-troubleshooting.md
 create mode 100644 .copilot-workflow.md
 create mode 100644 packages/duckdb/.copilot-package.md
 create mode 100644 packages/graphviz/.copilot-package.md

diff --git a/.copilot-instructions.md b/.copilot-instructions.md
new file mode 100644
index 00000000..cbed7259
--- /dev/null
+++ b/.copilot-instructions.md
@@ -0,0 +1,223 @@
+# Copilot Instructions for HPCC-JS-WASM
+
+This document provides guidance for AI assistants working with the HPCC-JS-WASM repository.
+
+## Repository Overview
+
+This is a **monorepo** that provides WebAssembly (WASM) versions of popular C++ libraries for use in Node.js, web browsers, and JavaScript applications. It uses **Lerna** for monorepo management and **npm workspaces**.
+
+### Packages Provided
+
+1. **@hpcc-js/wasm-base91** - Base91 encoding/decoding library
+2. **@hpcc-js/wasm-duckdb** - DuckDB embedded database  
+3. **@hpcc-js/wasm-expat** - Expat XML parser
+4. **@hpcc-js/wasm-graphviz** - Graphviz graph visualization library
+5. **@hpcc-js/wasm-graphviz-cli** - Command-line interface for Graphviz
+6. **@hpcc-js/wasm-llama** - Llama.cpp AI model library
+7. **@hpcc-js/wasm-zstd** - Zstandard compression library
+8. **@hpcc-js/wasm** - Meta package for backward compatibility
+
+## Architecture
+
+### File Structure
+```
+├── packages/           # Individual WASM packages
+│   ├── base91/        # Base91 package
+│   ├── duckdb/        # DuckDB package  
+│   ├── expat/         # Expat package
+│   ├── graphviz/      # Graphviz package
+│   ├── graphviz-cli/  # Graphviz CLI package
+│   ├── llama/         # Llama package
+│   ├── wasm/          # Meta package
+│   └── zstd/          # Zstd package
+├── src-cpp/           # Shared C++ source code
+├── scripts/           # Build and utility scripts
+├── docs/              # Documentation (VitePress)
+└── .vscode/           # VSCode configuration
+```
+
+### Package Structure Pattern
+Each package typically contains:
+```
+packages/[name]/
+├── src/               # TypeScript source
+├── src-cpp/           # C++ source code
+├── tests/             # Test files
+├── package.json       # Package configuration
+└── vitest.config.ts   # Test configuration
+```
+
+## Development Workflow
+
+### Prerequisites for Full Development
+```bash
+# Install dependencies
+npm ci
+
+# Install build dependencies (requires system tools)
+npm run install-build-deps  # Installs emsdk, vcpkg, playwright
+
+# Build C++ to WASM (requires emscripten)
+npm run build-cpp
+
+# Build TypeScript packages
+npm run build-ws
+```
+
+### Quick Development (TypeScript only)
+```bash
+# Install dependencies
+npm ci
+
+# Build only TypeScript (without C++ compilation)
+npm run build-ws
+
+# Run linting
+npm run lint
+
+# Note: Tests may fail without WASM builds
+```
+
+### Testing
+```bash
+# Run all tests (requires full build)
+npm run test
+
+# Run specific package tests
+cd packages/graphviz && npm test
+
+# Run browser tests
+npm run test-browser
+
+# Run node tests  
+npm run test-node
+```
+
+## Common Patterns
+
+### WASM Library Loading Pattern
+All WASM libraries follow this pattern:
+```typescript
+import { LibraryName } from "@hpcc-js/wasm-libraryname";
+
+// Async loading required
+const library = await LibraryName.load();
+
+// Use library methods
+const result = library.someMethod(input);
+```
+
+### Package Export Structure
+```typescript
+// src/index.ts - Main entry point
+export * from "./libraryname.ts";
+
+// src/libraryname.ts - Main implementation
+export class LibraryName {
+    static async load(): Promise<LibraryName> { ... }
+    someMethod(input: string): string { ... }
+    version(): string { ... }
+}
+```
+
+### Testing Patterns
+```typescript
+import { describe, it, expect } from "vitest";
+import { LibraryName } from "@hpcc-js/wasm-libraryname";
+
+describe("LibraryName", () => {
+    it("basic functionality", async () => {
+        const lib = await LibraryName.load();
+        const result = lib.someMethod("test");
+        expect(result).toBeDefined();
+    });
+});
+```
+
+## Key Technologies
+
+- **TypeScript** - Primary language for package implementations
+- **C++** - Source libraries compiled to WASM
+- **Emscripten** - C++ to WASM compilation toolchain
+- **Lerna** - Monorepo management
+- **Vitest** - Testing framework (with browser support)
+- **ESBuild** - TypeScript bundling
+- **VitePress** - Documentation generation
+
+## Build System
+
+### C++ Compilation Chain
+1. **vcpkg** - C++ package manager for dependencies
+2. **emscripten** - Compiles C++ to WASM
+3. **IDL files** - WebIDL bindings for C++ classes
+4. **Custom esbuild plugins** - Handle WASM imports
+
+### TypeScript Build
+1. **tsc** - Type generation
+2. **esbuild** - Bundling (ESM, CJS, UMD formats)
+3. **Custom plugins** - WASM asset handling
+
+## Common Issues and Solutions
+
+### Build Failures
+- **Missing WASM files**: Run `npm run build-cpp` first
+- **Type errors**: Ensure all packages are built with `npm run build-ws`
+- **Test failures in fresh clone**: Expected - requires full build process
+
+### Package Resolution Issues
+- Packages depend on each other being built
+- Use relative imports for local development
+- Run `lerna run build` to build all packages
+
+### WASM Loading Issues
+- WASM files must be accessible at runtime
+- Browser requires proper MIME types for .wasm files
+- Node.js needs appropriate file system access
+
+## Debugging
+
+### VSCode Configuration
+- Launch configurations provided for browser and Node.js debugging
+- Tasks configured for watch mode development
+- CMake integration for C++ development
+
+### Log Levels
+Most libraries support debug output:
+```typescript
+// Enable debug logging
+const lib = await LibraryName.load({ debug: true });
+```
+
+## Documentation
+
+- **API Docs**: Generated with TypeDoc at https://hpcc-systems.github.io/hpcc-js-wasm/
+- **Examples**: See `/examples` directory and HTML files in root
+- **Package READMEs**: Each package has specific documentation
+
+## AI Assistant Guidelines
+
+### When Making Changes
+1. **Understand the monorepo structure** - changes may affect multiple packages
+2. **Follow existing patterns** - each package follows similar structure
+3. **Test incrementally** - build and test after each significant change
+4. **Consider WASM implications** - changes to C++ require full rebuild
+5. **Update documentation** - maintain TypeDoc comments and READMEs
+
+### Safe Operations (No Full Build Required)
+- TypeScript code changes in `src/` directories
+- Test file modifications
+- Documentation updates
+- Package.json script modifications
+
+### Operations Requiring Full Build
+- C++ source code changes in `src-cpp/` directories
+- WebIDL (.idl) file changes
+- CMakeLists.txt modifications
+- New WASM library additions
+
+### Before Submitting Changes
+1. Run `npm run lint` to check code style
+2. Run `npm run build-ws` to ensure TypeScript builds
+3. If C++ was modified, run full build cycle
+4. Test affected packages individually
+5. Update relevant documentation
\ No newline at end of file
diff --git a/.copilot-patterns.md b/.copilot-patterns.md
new file mode 100644
index 00000000..49de0ac4
--- /dev/null
+++ b/.copilot-patterns.md
@@ -0,0 +1,259 @@
+# Package-Specific Patterns Quick Reference
+
+This document provides quick reference patterns for each WASM package in the repository.
+
+## Base91 Package (@hpcc-js/wasm-base91)
+
+### Purpose
+Base91 encoding/decoding for efficient binary-to-text conversion.
+
+### Common Usage
+```typescript
+import { Base91 } from "@hpcc-js/wasm-base91";
+
+const base91 = await Base91.load();
+const encoded = base91.encode(new Uint8Array([1, 2, 3, 4]));
+const decoded = base91.decode(encoded);
+```
+
+### Key Methods
+- `encode(data: Uint8Array): string` - Encode binary data to Base91 string
+- `decode(str: string): Uint8Array` - Decode Base91 string to binary data
+
+## DuckDB Package (@hpcc-js/wasm-duckdb)
+
+### Purpose
+Embedded SQL database for data analysis and queries.
+
+### Common Usage
+```typescript
+import { DuckDB } from "@hpcc-js/wasm-duckdb";
+
+const db = await DuckDB.load();
+await db.query("CREATE TABLE users (id INTEGER, name VARCHAR)");
+const result = await db.query("SELECT * FROM users");
+```
+
+### Key Methods
+- `query(sql: string): Promise<QueryResult>` - Execute SQL query
+- `insertJSON(table: string, data: object[]): Promise<void>` - Insert JSON data
+- `insertCSV(table: string, csv: string): Promise<void>` - Insert CSV data
+
+## Expat Package (@hpcc-js/wasm-expat)
+
+### Purpose
+Fast XML parsing for processing XML documents.
+
+### Common Usage
+```typescript
+import { Expat } from "@hpcc-js/wasm-expat";
+
+const expat = await Expat.load();
+const result = expat.parse('<root><item>value</item></root>');
+```
+
+### Key Methods
+- `parse(xml: string): object` - Parse XML string to JavaScript object
+- `validate(xml: string): boolean` - Validate XML syntax
+
+## Graphviz Package (@hpcc-js/wasm-graphviz)
+
+### Purpose
+Graph visualization using DOT language.
+
+### Common Usage
+```typescript
+import { Graphviz } from "@hpcc-js/wasm-graphviz";
+
+const graphviz = await Graphviz.load();
+const svg = graphviz.dot('digraph G { a -> b }');
+```
+
+### Key Methods
+- `dot(dot: string): string` - Render using dot engine (hierarchical)
+- `neato(dot: string): string` - Render using neato engine (force-directed)
+- `layout(dot: string, format: Format, engine: Engine): string` - Full control
+
+### Engine Types
+- **dot**: Hierarchical/directed graphs
+- **neato**: Undirected graphs, force-directed
+- **fdp**: Force-directed with edge lengths
+- **circo**: Circular layout
+- **twopi**: Radial layout
+
+## Llama Package (@hpcc-js/wasm-llama)
+
+### Purpose
+AI model inference using Llama.cpp.
+
+### Common Usage
+```typescript
+import { Llama } from "@hpcc-js/wasm-llama";
+
+const llama = await Llama.load();
+const response = await llama.generate("Hello, how are you?", {
+    maxTokens: 100
+});
+```
+
+### Key Methods
+- `loadModel(modelData: Uint8Array): Promise<void>` - Load model
+- `generate(prompt: string, options?: GenerateOptions): Promise<string>` - Generate text
+
+## Zstd Package (@hpcc-js/wasm-zstd)
+
+### Purpose
+Fast compression and decompression using Zstandard algorithm.
+
+### Common Usage
+```typescript
+import { Zstd } from "@hpcc-js/wasm-zstd";
+
+const zstd = await Zstd.load();
+const compressed = zstd.compress(data);
+const decompressed = zstd.decompress(compressed);
+```
+
+### Key Methods
+- `compress(data: Uint8Array, level?: number): Uint8Array` - Compress data
+- `decompress(data: Uint8Array): Uint8Array` - Decompress data
+
+## Meta Package (@hpcc-js/wasm)
+
+### Purpose
+Backward compatibility package that re-exports all other packages.
+
+### Common Usage
+```typescript
+// Import specific packages (recommended)
+import { Graphviz } from "@hpcc-js/wasm/graphviz";
+import { Zstd } from "@hpcc-js/wasm/zstd";
+
+// Or import all (backward compatibility)
+import { Graphviz, Zstd } from "@hpcc-js/wasm";
+```
+
+## Common Patterns Across All Packages
+
+### Async Loading Pattern
+```typescript
+// All packages follow this pattern
+const library = await LibraryName.load(options?);
+```
+
+### Error Handling
+```typescript
+try {
+    const library = await LibraryName.load();
+    const result = library.method(input);
+} catch (error) {
+    console.error("Library error:", error);
+}
+```
+
+### Version Information
+```typescript
+const library = await LibraryName.load();
+console.log("Version:", library.version());
+```
+
+### Memory Management
+```typescript
+// WASM instances handle memory automatically
+// No explicit cleanup required in JavaScript
+const library = await LibraryName.load();
+// Use library...
+// Memory freed when library goes out of scope
+```
+
+## Testing Patterns
+
+### Basic Test Structure
+```typescript
+import { describe, it, expect } from "vitest";
+import { LibraryName } from "@hpcc-js/wasm-packagename";
+
+describe("LibraryName", () => {
+    it("loads successfully", async () => {
+        const lib = await LibraryName.load();
+        expect(lib).toBeDefined();
+        expect(lib.version()).toBeDefined();
+    });
+
+    it("basic functionality", async () => {
+        const lib = await LibraryName.load();
+        const result = lib.mainMethod("test input");
+        expect(result).toBeDefined();
+    });
+});
+```
+
+### Browser vs Node.js Tests
+```typescript
+// Node.js specific test
+import { describe, it, expect } from "vitest";
+
+// Browser specific test (filename: *.browser.spec.ts)
+describe("Browser environment", () => {
+    it("works in browser", async () => {
+        // Browser-specific logic
+    });
+});
+```
+
+## Performance Characteristics
+
+### Loading Time
+- **Fastest**: Base91, Expat
+- **Medium**: Zstd, Graphviz
+- **Slowest**: DuckDB, Llama (large WASM files)
+
+### Memory Usage
+- **Light**: Base91, Expat
+- **Medium**: Zstd, Graphviz
+- **Heavy**: DuckDB, Llama
+
+### Use Case Suitability
+- **Real-time**: Base91, Zstd, Expat
+- **Interactive**: Graphviz
+- **Batch processing**: DuckDB, Llama
+
+## Integration Examples
+
+### Compression Pipeline
+```typescript
+const zstd = await Zstd.load();
+const base91 = await Base91.load();
+
+// Compress and encode
+const compressed = zstd.compress(data);
+const encoded = base91.encode(compressed);
+
+// Decode and decompress
+const decoded = base91.decode(encoded);
+const decompressed = zstd.decompress(decoded);
+```
+
+### Data Visualization
+```typescript
+const duckdb = await DuckDB.load();
+const graphviz = await Graphviz.load();
+
+// Query data
+const results = await duckdb.query("SELECT * FROM relationships");
+
+// Generate graph
+const dot = generateDotFromResults(results);
+const svg = graphviz.dot(dot);
+```
+
+### Document Processing
+```typescript
+const expat = await Expat.load();
+const zstd = await Zstd.load();
+
+// Parse and compress
+const parsed = expat.parse(xmlDocument);
+const serialized = JSON.stringify(parsed);
+const compressed = zstd.compress(new TextEncoder().encode(serialized));
+```
\ No newline at end of file
diff --git a/.copilot-troubleshooting.md b/.copilot-troubleshooting.md
new file mode 100644
index 00000000..28c7ac55
--- /dev/null
+++ b/.copilot-troubleshooting.md
@@ -0,0 +1,425 @@
+# AI Assistant Troubleshooting Guide
+
+This guide helps AI assistants diagnose and resolve common issues in the HPCC-JS-WASM repository.
+
+## Quick Diagnosis Commands
+
+### Check Repository State
+```bash
+# Basic health check
+git status
+npm ls --depth=0
+ls -la packages/*/dist/ 2>/dev/null | wc -l  # Count built packages
+
+# Check for common build artifacts
+find . -name "*.wasm" -not -path "./node_modules/*" | head -5
+find . -name "*.d.ts" -not -path "./node_modules/*" | head -5
+```
+
+### Environment Check
+```bash
+# Node.js and npm versions
+node --version
+npm --version
+
+# Check if build tools are available
+which emcc 2>/dev/null && echo "Emscripten available" || echo "Emscripten not found"
+ls -d emsdk 2>/dev/null && echo "emsdk installed" || echo "emsdk not found"
+ls -d vcpkg 2>/dev/null && echo "vcpkg installed" || echo "vcpkg not found"
+```
+
+## Common Error Patterns and Solutions
+
+### 1. "Failed to resolve entry for package" Error
+
+**Symptoms:**
+```
+Error: Failed to resolve entry for package "@hpcc-js/wasm-graphviz". 
+The package may have incorrect main/module/exports specified in its package.json.
+```
+
+**Diagnosis:**
+```bash
+# Check if package is built
+ls packages/graphviz/dist/
+ls packages/graphviz/types/
+
+# Check package.json exports
+cat packages/graphviz/package.json | grep -A 10 '"exports"'
+```
+
+**Solutions:**
+```bash
+# Solution 1: Build the specific package
+cd packages/graphviz
+npm run build
+
+# Solution 2: Build all packages
+npm run build-ws
+
+# Solution 3: Clean and rebuild
+npm run clean
+npm ci
+npm run build-ws
+```
+
+### 2. "Executable doesn't exist" (Playwright/Browser Tests)
+
+**Symptoms:**
+```
+Error: browserType.launch: Executable doesn't exist at /home/runner/.cache/ms-playwright/chromium_headless_shell-1179/chrome-linux/headless_shell
+```
+
+**Solutions:**
+```bash
+# Install Playwright browsers
+npx playwright install
+
+# Or with system dependencies
+npx playwright install --with-deps
+
+# Alternative: Skip browser tests
+npm run test-node  # Run only Node.js tests
+```
+
+### 3. WASM Loading Failures
+
+**Symptoms:**
+```
+Error: Module not found: *.wasm
+TypeError: Cannot read properties of undefined (loading WASM)
+```
+
+**Diagnosis:**
+```bash
+# Check if WASM files exist
+find build/ -name "*.wasm" 2>/dev/null
+ls packages/*/src-cpp/*.wasm 2>/dev/null
+```
+
+**Solutions:**
+```bash
+# Solution 1: Build C++ to WASM
+npm run build-cpp
+
+# Solution 2: Use Docker build (if local tools missing)
+npm run build-docker
+
+# Solution 3: Check build dependencies
+npm run install-build-deps
+```
+
+### 4. TypeScript Type Errors
+
+**Symptoms:**
+```
+TS2307: Cannot find module '@hpcc-js/wasm-graphviz' or its corresponding type declarations
+```
+
+**Diagnosis:**
+```bash
+# Check if types are generated
+ls packages/graphviz/types/
+cat packages/graphviz/package.json | grep '"types"'
+```
+
+**Solutions:**
+```bash
+# Generate types for specific package
+cd packages/graphviz
+npm run gen-types
+
+# Generate types for all packages
+npm run build-ws
+
+# Check TypeScript configuration
+npx tsc --noEmit
+```
+
+### 5. Lerna/Workspace Issues
+
+**Symptoms:**
+```
+lerna ERR! ENOPACKAGES No packages found
+npm ERR! Missing script: "build"
+```
+
+**Diagnosis:**
+```bash
+# Check lerna configuration
+cat lerna.json
+cat package.json | grep -A 5 '"workspaces"'
+
+# List packages lerna can see
+npx lerna ls
+```
+
+**Solutions:**
+```bash
+# Reinstall dependencies
+npm ci
+
+# Bootstrap lerna
+npx lerna bootstrap
+
+# Clean and reinstall
+npm run clean-all
+npm ci
+```
+
+### 6. Memory Issues (Browser)
+
+**Symptoms:**
+```
+RangeError: Maximum call stack size exceeded
+Out of memory errors in browser
+```
+
+**Solutions:**
+```bash
+# Use smaller test datasets
+# Reduce concurrent operations
+# Test in Node.js instead of browser
+npm run test-node
+
+# Check memory usage patterns in code
+grep -r "new Uint8Array" packages/*/src/
+```
+
+### 7. Import/Export Module Issues
+
+**Symptoms:**
+```
+SyntaxError: Cannot use import statement outside a module
+Module format mismatch errors
+```
+
+**Diagnosis:**
+```bash
+# Check module types in package.json
+grep -r '"type"' packages/*/package.json
+grep -r '"module"' packages/*/package.json
+
+# Check built file formats
+ls packages/*/dist/*.{js,cjs,mjs} 2>/dev/null
+```
+
+**Solutions:**
+```bash
+# Ensure proper build
+npm run build-ws
+
+# Check esbuild configuration
+cat packages/*/esbuild.js 2>/dev/null | head -20
+```
+
+## Debugging Workflows
+
+### For TypeScript-Only Issues
+```bash
+# 1. Check syntax
+npx eslint packages/*/src/**/*.ts
+
+# 2. Check types
+npx tsc --noEmit
+
+# 3. Build incrementally
+cd packages/[problematic-package]
+npm run gen-types
+npm run bundle
+
+# 4. Test specific package
+npm test
+```
+
+### For WASM-Related Issues
+```bash
+# 1. Check C++ build tools
+which emcc || echo "Need emscripten"
+ls emsdk/ || echo "Need emsdk"
+
+# 2. Try minimal C++ build
+cd packages/base91  # Smallest package
+npm run build
+
+# 3. Check WASM loading
+node -e "
+const fs = require('fs');
+const wasm = fs.readFileSync('build/packages/base91/src-cpp/base91lib.wasm');
+console.log('WASM size:', wasm.length);
+"
+```
+
+### For Test Failures
+```bash
+# 1. Run specific test file
+cd packages/[package]
+npx vitest run tests/[specific-test].spec.ts
+
+# 2. Run with verbose output
+npx vitest run --reporter=verbose
+
+# 3. Run in different environments
+npm run test-node    # Node.js only
+npm run test-browser # Browser only
+
+# 4. Check test configuration
+cat vitest.config.ts
+```
+
+## Performance Troubleshooting
+
+### Slow Builds
+```bash
+# Check what's taking time
+time npm run build-cpp
+time npm run build-ws
+
+# Use watch modes for development
+npm run build-ts-watch &
+npm run build-cpp-watch &
+```
+
+### Memory Usage
+```bash
+# Monitor memory during tests
+top -p $(pgrep node)
+
+# Use Node.js memory profiling
+node --max-old-space-size=4096 node_modules/.bin/vitest
+```
+
+### Bundle Size Issues
+```bash
+# Check bundle sizes
+ls -lh packages/*/dist/*.js
+
+# Analyze bundle content
+npx esbuild packages/[package]/src/index.ts --bundle --analyze
+```
+
+## Environment-Specific Issues
+
+### Windows (WSL) Issues
+```bash
+# Check WSL integration
+wsl --version
+echo $WSL_DISTRO_NAME
+
+# Use WSL for build scripts
+npm run build-cpp:win32
+```
+
+### Docker Issues
+```bash
+# Check Docker setup
+docker --version
+docker images | grep hpcc-js-wasm
+
+# Build in Docker
+npm run build-docker-image
+npm run build-docker-wasm
+```
+
+### CI/CD Issues
+```bash
+# Check GitHub Actions
+cat .github/workflows/test-pr.yml
+
+# Local CI simulation
+act  # If nektos/act is available
+```
+
+## Recovery Procedures
+
+### Complete Reset
+```bash
+# Nuclear option - start fresh
+npm run uninstall-build-deps
+npm run clean-all
+npm ci
+npm run install-build-deps
+npm run build
+```
+
+### Partial Reset
+```bash
+# Reset specific package
+cd packages/[package]
+npm run clean
+npm run build
+
+# Reset build artifacts only
+npm run clean-build
+npm run build-ws
+```
+
+### Git Reset
+```bash
+# Undo local changes
+git checkout -- .
+git clean -fd
+
+# Reset to last commit
+git reset --hard HEAD
+```
+
+## Preventive Measures
+
+### Before Making Changes
+1. **Save current state**: `git stash`
+2. **Check tests pass**: `npm run lint && npm run build-ws`
+3. **Create branch**: `git checkout -b feature/my-changes`
+
+### During Development
+1. **Test incrementally**: Build and test after each change
+2. **Watch for errors**: Use watch modes (`npm run build-ts-watch`)
+3. **Monitor memory**: Keep an eye on process memory usage
+
+### Before Committing
+1. **Full lint**: `npm run lint`
+2. **Type check**: `npx tsc --noEmit`
+3. **Test affected packages**: Focus on changed packages
+4. **Clean build**: `npm run clean && npm run build-ws`
+
+## When to Ask for Help
+
+### Escalate if:
+1. **System dependencies missing**: emscripten, cmake, etc.
+2. **Platform-specific issues**: Windows, specific Linux distros
+3. **Memory/performance issues**: Beyond normal optimization
+4. **C++ compilation errors**: Deep WASM/emscripten issues
+5. **Network/dependency issues**: npm, GitHub API problems
+
+### Provide This Information:
+```bash
+# System info
+uname -a
+node --version
+npm --version
+
+# Repository state
+git status
+git log --oneline -3
+npm ls --depth=0
+
+# Error details
+# Include full error messages and stack traces
+# Include command that caused the error
+# Include any recent changes made
+```
+
+## Quick Fixes Cheat Sheet
+
+| Error | Quick Fix |
+|-------|-----------|
+| Package not found | `npm run build-ws` |
+| WASM loading error | `npm run build-cpp` |
+| Type errors | `cd packages/[pkg] && npm run gen-types` |
+| Test failures | `npx playwright install` |
+| Linting errors | `npm run lint-fix` |
+| Memory issues | Use Node.js tests: `npm run test-node` |
+| Import errors | Check package.json exports and rebuild |
+| Docker issues | `npm run build-docker` |
+
+Remember: Most issues in this repository stem from the complex build process. When in doubt, try a clean rebuild: `npm run clean && npm ci && npm run build-ws`
\ No newline at end of file
diff --git a/.copilot-workflow.md b/.copilot-workflow.md
new file mode 100644
index 00000000..d17a9e5a
--- /dev/null
+++ b/.copilot-workflow.md
@@ -0,0 +1,373 @@
+# AI Assistant Development Workflow
+
+This guide provides step-by-step workflows for common development tasks when working with the HPCC-JS-WASM repository.
+
+## Initial Setup and Assessment
+
+### 1. Fresh Repository Assessment
+```bash
+# Check repository state
+git status
+git log --oneline -5
+
+# Check if dependencies are installed
+ls node_modules/ || echo "Dependencies not installed"
+
+# Check if packages are built
+ls packages/*/dist/ || echo "Packages not built"
+
+# Install dependencies if needed
+npm ci
+```
+
+### 2. Quick Health Check
+```bash
+# Test linting (should work without builds)
+npm run lint
+
+# Try building TypeScript only (may fail without WASM)
+npm run build-ws
+
+# Check individual package status
+cd packages/graphviz && npm run gen-types
+```
+
+## Development Workflows
+
+### Workflow 1: TypeScript-Only Changes
+For changes to TypeScript code, tests, or documentation:
+
+```bash
+# 1. Install dependencies
+npm ci
+
+# 2. Build TypeScript packages
+npm run build-ws
+
+# 3. Run linting
+npm run lint
+
+# 4. Test specific package
+cd packages/[package-name]
+npm test
+
+# 5. Fix any linting issues
+npm run lint-fix
+```
+
+### Workflow 2: Full Development Setup
+For comprehensive development including C++ changes:
+
+```bash
+# 1. Install all dependencies
+npm ci
+
+# 2. Install build tools (requires system dependencies)
+npm run install-build-deps
+
+# 3. Build C++ to WASM
+npm run build-cpp
+
+# 4. Build TypeScript packages
+npm run build-ws
+
+# 5. Run full test suite
+npm run test
+```
+
+### Workflow 3: Docker-Based Development
+For consistent environment without local setup:
+
+```bash
+# Build and test in Docker
+npm run build-docker
+```
+
+## Common Development Tasks
+
+### Adding a New Method to a Package
+
+1. **Identify the target package**
+```bash
+cd packages/[package-name]
+```
+
+2. **Understand the current API**
+```typescript
+// Look at src/index.ts and main class file
+cat src/index.ts
+cat src/[package-name].ts
+```
+
+3. **Add the new method**
+```typescript
+// In src/[package-name].ts
+export class PackageName {
+    // ... existing methods
+
+    newMethod(input: string): string {
+        // Implementation
+        return this._wasmInstance.callMethod(input);
+    }
+}
+```
+
+4. **Add TypeScript types if needed**
+```typescript
+// Add to type definitions
+export interface NewMethodOptions {
+    option1?: string;
+    option2?: number;
+}
+```
+
+5. **Add tests**
+```typescript
+// In tests/[package-name].spec.ts
+it("new method works", async () => {
+    const lib = await PackageName.load();
+    const result = lib.newMethod("test");
+    expect(result).toBeDefined();
+});
+```
+
+6. **Build and test**
+```bash
+npm run build
+npm test
+```
+
+### Fixing a Bug
+
+1. **Reproduce the issue**
+```bash
+# Write a failing test first
+npm test -- --reporter=verbose
+```
+
+2. **Identify the root cause**
+```typescript
+// Add debug logging
+console.log("Debug info:", variable);
+```
+
+3. **Make minimal fix**
+```typescript
+// Fix the specific issue
+if (condition) {
+    // Handle edge case
+}
+```
+
+4. **Verify fix**
+```bash
+npm test
+npm run lint
+```
+
+### Adding New Tests
+
+1. **Identify test type needed**
+- Unit tests: `tests/[package].spec.ts`
+- Browser tests: `tests/[package].browser.spec.ts`
+- Worker tests: `tests/worker.*.spec.ts`
+
+2. **Follow existing patterns**
+```typescript
+import { describe, it, expect } from "vitest";
+import { PackageName } from "@hpcc-js/wasm-packagename";
+
+describe("Feature", () => {
+    it("should do something", async () => {
+        const lib = await PackageName.load();
+        // Test implementation
+    });
+});
+```
+
+3. **Run tests**
+```bash
+npm test
+```
+
+## Troubleshooting Workflows
+
+### Build Failures
+
+1. **Clean and rebuild**
+```bash
+npm run clean-all
+npm ci
+npm run build-ws
+```
+
+2. **Check specific package**
+```bash
+cd packages/[failing-package]
+npm run clean
+npm run build
+```
+
+3. **Check dependencies**
+```bash
+npm ls
+lerna ls
+```
+
+### Test Failures
+
+1. **Run specific test**
+```bash
+cd packages/[package]
+npm test -- --reporter=verbose
+```
+
+2. **Check if WASM files exist**
+```bash
+ls build/packages/*/src-cpp/*.wasm
+```
+
+3. **Run without browser tests**
+```bash
+npm run test-node
+```
+
+### Import/Export Issues
+
+1. **Check package exports**
+```bash
+cat packages/[package]/package.json | grep -A 10 '"exports"'
+```
+
+2. **Verify built files exist**
+```bash
+ls packages/[package]/dist/
+ls packages/[package]/types/
+```
+
+3. **Check TypeScript compilation**
+```bash
+cd packages/[package]
+npm run gen-types
+```
+
+## Code Quality Workflows
+
+### Before Committing Changes
+
+1. **Run linting**
+```bash
+npm run lint
+npm run lint-fix
+```
+
+2. **Build all packages**
+```bash
+npm run build-ws
+```
+
+3. **Test affected packages**
+```bash
+# Test specific packages
+cd packages/[modified-package]
+npm test
+```
+
+4. **Check types**
+```bash
+npx tsc --noEmit
+```
+
+### Documentation Updates
+
+1. **Update TypeDoc comments**
+```typescript
+/**
+ * Description of the method
+ * @param input - Description of parameter
+ * @returns Description of return value
+ * @example
+ * ```ts
+ * const result = lib.method("example");
+ * ```
+ */
+method(input: string): string { ... }
+```
+
+2. **Update README if needed**
+```bash
+# Check if package README needs updates
+cat packages/[package]/README.md
+```
+
+3. **Generate documentation**
+```bash
+npm run gen-docs
+```
+
+## Release Workflow
+
+### Version Bumping
+```bash
+# Update dependencies
+npm run update
+
+# Use lerna for version management
+lerna version --conventional-commits
+```
+
+### Publishing
+```bash
+# Build everything
+npm run build
+
+# Publish via lerna
+lerna publish from-package
+```
+
+## Emergency Fixes
+
+### Reverting Changes
+```bash
+# Revert specific files
+git checkout HEAD -- packages/[package]/src/[file].ts
+
+# Revert last commit
+git revert HEAD
+```
+
+### Quick Package Fix
+```bash
+# Fix specific package quickly
+cd packages/[package]
+npm run clean
+npm run build
+npm test
+```
+
+## Performance Considerations
+
+### Memory Usage
+- Large WASM files consume significant memory
+- Test with realistic data sizes
+- Monitor memory usage in browser dev tools
+
+### Build Time
+- C++ compilation is slowest part
+- TypeScript builds are relatively fast
+- Use watch modes for development
+
+### Testing Speed
+- Browser tests are slower than Node.js tests
+- Run specific test suites during development
+- Use CI for comprehensive testing
+
+## AI Assistant Best Practices
+
+1. **Always check current state** before making changes
+2. **Test incrementally** after each change
+3. **Follow existing patterns** rather than creating new ones
+4. **Keep changes minimal** and focused
+5. **Update documentation** when changing APIs
+6. **Consider backward compatibility** for public APIs
+7. **Test in both browser and Node.js** environments when applicable
\ No newline at end of file
diff --git a/.vscode/settings.json b/.vscode/settings.json
index bb6b3c91..7623d70e 100644
--- a/.vscode/settings.json
+++ b/.vscode/settings.json
@@ -1,5 +1,40 @@
 {
     "cmake.cmakePath": "${workspaceFolder}/scripts/cmake.sh",
     "vitest.maximumConfigs": 20,
-    "vitest.configSearchPatternExclude": "{emsdk/**,node_modules/**}"
+    "vitest.configSearchPatternExclude": "{emsdk/**,node_modules/**}",
+    
+    // AI Assistant Configuration
+    "github.copilot.enable": {
+        "*": true,
+        "markdown": true,
+        "typescript": true,
+        "javascript": true
+    },
+    
+    // File associations for better AI context
+    "files.associations": {
+        "*.idl": "webidl",
+        "*.wasm": "wasm",
+        ".copilot-*": "markdown"
+    },
+    
+    // Exclude build artifacts from AI context
+    "files.exclude": {
+        "**/build/**": true,
+        "**/dist/**": true,
+        "**/node_modules/**": true,
+        "**/emsdk/**": true,
+        "**/vcpkg/**": true,
+        "**/.nyc_output/**": true
+    },
+    
+    // Include important files for AI context
+    "search.exclude": {
+        "**/node_modules": true,
+        "**/emsdk": true,
+        "**/vcpkg": true,
+        "**/build": true,
+        "**/dist": false,
+        "**/.copilot-*": false
+    }
 }
\ No newline at end of file
diff --git a/README.md b/README.md
index c1225959..f9274154 100644
--- a/README.md
+++ b/README.md
@@ -16,15 +16,25 @@ This repository contains a collection of useful c++ libraries compiled to WASM f
 Built with:
 - [emsdk](https://github.com/emscripten-core/emsdk) - v4.0.10
 
-## Homepage and Documents
-
-* [Homepage](https://hpcc-systems.github.io/hpcc-js-wasm/)
-    * [Base91](https://hpcc-systems.github.io/hpcc-js-wasm/base91/src/base91/classes/Base91.html)
-    * [DuckDB](https://hpcc-systems.github.io/hpcc-js-wasm/duckdb/src/duckdb/classes/DuckDB.html)
-    * [Expat](https://hpcc-systems.github.io/hpcc-js-wasm/expat/src/expat/classes/Expat.html)
-    * [Graphviz](https://hpcc-systems.github.io/hpcc-js-wasm/graphviz/src/graphviz/classes/Graphviz.html)
-    * [Llama](https://hpcc-systems.github.io/hpcc-js-wasm/llama/src/llama/classes/Llama.html)
-    * [Zstd](https://hpcc-systems.github.io/hpcc-js-wasm/zstd/src/zstd/classes/Zstd.html)
+## Homepage and Documents
+
+* [Homepage](https://hpcc-systems.github.io/hpcc-js-wasm/)
+    * [Base91](https://hpcc-systems.github.io/hpcc-js-wasm/base91/src/base91/classes/Base91.html)
+    * [DuckDB](https://hpcc-systems.github.io/hpcc-js-wasm/duckdb/src/duckdb/classes/DuckDB.html)
+    * [Expat](https://hpcc-systems.github.io/hpcc-js-wasm/expat/src/expat/classes/Expat.html)
+    * [Graphviz](https://hpcc-systems.github.io/hpcc-js-wasm/graphviz/src/graphviz/classes/Graphviz.html)
+    * [Llama](https://hpcc-systems.github.io/hpcc-js-wasm/llama/src/llama/classes/Llama.html)
+    * [Zstd](https://hpcc-systems.github.io/hpcc-js-wasm/zstd/src/zstd/classes/Zstd.html)
+
+## AI Assistant / Copilot Instructions
+
+This repository includes comprehensive instructions for AI assistants and GitHub Copilot:
+
+* [Main Copilot Instructions](.copilot-instructions.md) - Repository overview and architecture
+* [Development Workflow](.copilot-workflow.md) - Step-by-step development processes
+* [Package Patterns](.copilot-patterns.md) - Common patterns and quick reference
+* [Troubleshooting Guide](.copilot-troubleshooting.md) - Debugging and problem resolution
+* Package-specific guides in `packages/*/` directories
 
 ## Version 3 Changes
 
diff --git a/packages/duckdb/.copilot-package.md b/packages/duckdb/.copilot-package.md
new file mode 100644
index 00000000..b1693013
--- /dev/null
+++ b/packages/duckdb/.copilot-package.md
@@ -0,0 +1,398 @@
+# DuckDB Package - AI Assistant Guidelines
+
+This document provides specific guidance for working with the `@hpcc-js/wasm-duckdb` package.
+
+## Package Overview
+
+Provides WebAssembly version of [DuckDB](https://duckdb.org/) - an embedded analytical database designed for complex analytical queries.
+
+### Key Features
+- **In-Memory SQL Database**: Fast analytical queries without server setup
+- **JSON/CSV Import**: Direct import from common data formats
+- **Standard SQL**: Full SQL support with advanced analytical functions
+- **Parquet Support**: Read/write Parquet files
+- **Browser & Node.js Compatible**: Works in both environments
+
+## Package Structure
+
+```
+packages/duckdb/
+├── src/
+│   ├── index.ts          # Package entry point
+│   ├── duckdb.ts         # Main DuckDB class implementation
+│   └── __package__.ts    # Package metadata
+├── src-cpp/              # C++ DuckDB source (large)
+├── tests/                # Test files
+│   ├── duckdb.spec.ts    # Main tests
+│   └── duckdb.browser.spec.ts  # Browser-specific tests
+└── package.json          # Package configuration
+```
+
+## API Overview
+
+### Main Class
+```typescript
+export class DuckDB {
+    static async load(): Promise<DuckDB>
+    
+    // Core query methods
+    query(sql: string): Promise<QueryResult>
+    prepare(sql: string): Promise<PreparedStatement>
+    
+    // Data import methods
+    insertJSON(tableName: string, json: object[]): Promise<void>
+    insertCSV(tableName: string, csv: string, options?: CSVOptions): Promise<void>
+    insertParquet(tableName: string, parquet: Uint8Array): Promise<void>
+    
+    // Data export methods
+    exportJSON(sql: string): Promise<object[]>
+    exportCSV(sql: string): Promise<string>
+    exportParquet(sql: string): Promise<Uint8Array>
+    
+    // Utility methods
+    version(): string
+    listTables(): Promise<string[]>
+    describeTable(tableName: string): Promise<ColumnInfo[]>
+}
+```
+
+### Types and Interfaces
+```typescript
+interface QueryResult {
+    columns: string[];
+    data: any[][];
+    rowCount: number;
+}
+
+interface PreparedStatement {
+    execute(...params: any[]): Promise<QueryResult>
+    close(): void
+}
+
+interface CSVOptions {
+    delimiter?: string;
+    quote?: string;
+    header?: boolean;
+    nullstr?: string;
+}
+
+interface ColumnInfo {
+    name: string;
+    type: string;
+    nullable: boolean;
+}
+```
+
+## Usage Patterns
+
+### Basic SQL Operations
+```typescript
+import { DuckDB } from "@hpcc-js/wasm-duckdb";
+
+const db = await DuckDB.load();
+
+// Create table
+await db.query(`
+    CREATE TABLE users (
+        id INTEGER PRIMARY KEY,
+        name VARCHAR,
+        age INTEGER,
+        email VARCHAR
+    )
+`);
+
+// Insert data
+await db.query(`
+    INSERT INTO users VALUES 
+    (1, 'Alice', 30, 'alice@example.com'),
+    (2, 'Bob', 25, 'bob@example.com')
+`);
+
+// Query data
+const result = await db.query("SELECT * FROM users WHERE age > 25");
+console.log(result);
+```
+
+### JSON Data Import
+```typescript
+const users = [
+    { id: 1, name: "Alice", age: 30, email: "alice@example.com" },
+    { id: 2, name: "Bob", age: 25, email: "bob@example.com" }
+];
+
+await db.insertJSON("users", users);
+const result = await db.query("SELECT AVG(age) as avg_age FROM users");
+```
+
+### CSV Data Processing
+```typescript
+const csvData = `name,age,city
+Alice,30,New York
+Bob,25,Los Angeles
+Charlie,35,Chicago`;
+
+await db.insertCSV("people", csvData, { header: true });
+const result = await db.query(`
+    SELECT city, COUNT(*) as count 
+    FROM people 
+    GROUP BY city 
+    ORDER BY count DESC
+`);
+```
+
+### Analytical Queries
+```typescript
+// Window functions
+const salesAnalysis = await db.query(`
+    SELECT 
+        month,
+        revenue,
+        SUM(revenue) OVER (ORDER BY month) as running_total,
+        LAG(revenue) OVER (ORDER BY month) as prev_month,
+        revenue - LAG(revenue) OVER (ORDER BY month) as growth
+    FROM monthly_sales
+    ORDER BY month
+`);
+
+// Common Table Expressions (CTEs)
+const complexQuery = await db.query(`
+    WITH user_stats AS (
+        SELECT user_id, COUNT(*) as order_count, SUM(amount) as total_spent
+        FROM orders
+        GROUP BY user_id
+    )
+    SELECT u.name, us.order_count, us.total_spent
+    FROM users u
+    JOIN user_stats us ON u.id = us.user_id
+    WHERE us.total_spent > 1000
+`);
+```
+
+### Prepared Statements
+```typescript
+const stmt = await db.prepare("SELECT * FROM users WHERE age > ? AND city = ?");
+
+const youngUsers = await stmt.execute(25, "New York");
+const olderUsers = await stmt.execute(35, "Chicago");
+
+stmt.close(); // Clean up resources
+```
+
+## Testing Patterns
+
+### Basic Database Tests
+```typescript
+import { describe, it, expect, beforeEach } from "vitest";
+import { DuckDB } from "@hpcc-js/wasm-duckdb";
+
+describe("DuckDB", () => {
+    let db: DuckDB;
+
+    beforeEach(async () => {
+        db = await DuckDB.load();
+    });
+
+    it("creates and queries table", async () => {
+        await db.query("CREATE TABLE test (id INTEGER, value VARCHAR)");
+        await db.query("INSERT INTO test VALUES (1, 'hello')");
+        
+        const result = await db.query("SELECT * FROM test");
+        expect(result.data).toHaveLength(1);
+        expect(result.data[0]).toEqual([1, 'hello']);
+    });
+});
+```
+
+### Data Import Tests
+```typescript
+it("imports JSON data", async () => {
+    const data = [{ id: 1, name: "test" }];
+    await db.insertJSON("test_table", data);
+    
+    const result = await db.query("SELECT COUNT(*) as count FROM test_table");
+    expect(result.data[0][0]).toBe(1);
+});
+
+it("imports CSV with options", async () => {
+    const csv = "id|name\n1|test";
+    await db.insertCSV("test_table", csv, { delimiter: "|", header: true });
+    
+    const result = await db.query("SELECT name FROM test_table WHERE id = 1");
+    expect(result.data[0][0]).toBe("test");
+});
+```
+
+## Performance Considerations
+
+### Memory Management
+- DuckDB uses significant memory for large datasets
+- Consider memory limits in browser environments
+- Use LIMIT clauses for large result sets
+
+### Query Optimization
+```typescript
+// Good: Use specific columns
+const efficient = await db.query("SELECT id, name FROM large_table WHERE id = 1");
+
+// Avoid: SELECT * on large tables
+const inefficient = await db.query("SELECT * FROM large_table");
+
+// Good: Use indexes
+await db.query("CREATE INDEX idx_user_id ON orders(user_id)");
+```
+
+### Batch Operations
+```typescript
+// Good: Batch inserts
+await db.insertJSON("table", largeArray);
+
+// Avoid: Individual inserts in loop
+for (const item of largeArray) {
+    await db.query("INSERT INTO table VALUES (?, ?)", item.id, item.value);
+}
+```
+
+## Common SQL Patterns
+
+### Data Analysis
+```typescript
+// Aggregations with grouping
+const summary = await db.query(`
+    SELECT 
+        category,
+        COUNT(*) as item_count,
+        AVG(price) as avg_price,
+        SUM(price) as total_value
+    FROM products
+    GROUP BY category
+    HAVING COUNT(*) > 10
+    ORDER BY total_value DESC
+`);
+
+// Percentiles and statistics
+const stats = await db.query(`
+    SELECT 
+        PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY price) as median_price,
+        STDDEV(price) as price_stddev,
+        MIN(price) as min_price,
+        MAX(price) as max_price
+    FROM products
+`);
+```
+
+### Time Series Analysis
+```typescript
+// Date functions and windows
+const timeSeries = await db.query(`
+    SELECT 
+        DATE_TRUNC('month', order_date) as month,
+        SUM(amount) as monthly_total,
+        AVG(SUM(amount)) OVER (
+            ORDER BY DATE_TRUNC('month', order_date) 
+            ROWS BETWEEN 2 PRECEDING AND CURRENT ROW
+        ) as moving_avg
+    FROM orders
+    GROUP BY DATE_TRUNC('month', order_date)
+    ORDER BY month
+`);
+```
+
+### Data Cleaning
+```typescript
+// Handle missing values and duplicates
+const cleaned = await db.query(`
+    SELECT DISTINCT
+        COALESCE(name, 'Unknown') as name,
+        CASE 
+            WHEN age < 0 OR age > 120 THEN NULL 
+            ELSE age 
+        END as age,
+        TRIM(LOWER(email)) as email
+    FROM raw_users
+    WHERE email IS NOT NULL AND email != ''
+`);
+```
+
+## Browser vs Node.js Considerations
+
+### Browser Limitations
+- Memory constraints with large datasets
+- File system access limitations
+- Performance may be slower than Node.js
+
+### Node.js Advantages
+- Better memory management
+- File system access for data import/export
+- Generally better performance
+
+### Cross-Platform Code
+```typescript
+// Feature detection
+const isNode = typeof process !== 'undefined' && process.versions?.node;
+
+if (isNode) {
+    // Node.js specific optimizations
+    console.log("Running in Node.js - can handle larger datasets");
+} else {
+    // Browser specific considerations
+    console.log("Running in browser - memory constraints apply");
+}
+```
+
+## Error Handling
+
+### Common Error Patterns
+```typescript
+try {
+    const result = await db.query("SELECT * FROM non_existent_table");
+} catch (error) {
+    if (error.message.includes("does not exist")) {
+        console.log("Table not found - creating it");
+        await db.query("CREATE TABLE non_existent_table (id INTEGER)");
+    } else {
+        throw error; // Re-throw unexpected errors
+    }
+}
+```
+
+### SQL Validation
+```typescript
+function validateSQL(sql: string): boolean {
+    // Basic SQL injection prevention
+    const dangerous = /\b(DROP|DELETE|UPDATE|INSERT|ALTER|CREATE)\b/i;
+    if (dangerous.test(sql) && !sql.includes("--safe")) {
+        throw new Error("Potentially dangerous SQL detected");
+    }
+    return true;
+}
+```
+
+## AI Assistant Guidelines
+
+### When Modifying This Package
+1. **Understand SQL**: Familiarize yourself with SQL syntax and DuckDB extensions
+2. **Memory awareness**: Consider memory usage for large datasets
+3. **Cross-platform compatibility**: Test in both browser and Node.js
+4. **Performance**: Use appropriate query patterns and indexes
+5. **Error handling**: Handle database errors gracefully
+
+### Safe Modifications
+- Adding new query helper methods
+- Improving TypeScript types
+- Adding new data import/export formats
+- Performance optimizations for queries
+- Better error messages
+
+### Modifications Requiring C++ Changes
+- Adding new DuckDB features not in current version
+- Custom functions or aggregates
+- Performance optimizations at database level
+- Memory management improvements
+
+### Testing Requirements
+- Test with various SQL constructs and data types
+- Test data import/export functionality
+- Test error handling with invalid SQL
+- Test memory usage with large datasets
+- Test both browser and Node.js environments
+- Performance benchmarks for complex queries
\ No newline at end of file
diff --git a/packages/graphviz/.copilot-package.md b/packages/graphviz/.copilot-package.md
new file mode 100644
index 00000000..f3bbefee
--- /dev/null
+++ b/packages/graphviz/.copilot-package.md
@@ -0,0 +1,245 @@
+# Graphviz Package - AI Assistant Guidelines
+
+This document provides specific guidance for working with the `@hpcc-js/wasm-graphviz` package.
+
+## Package Overview
+
+Provides WebAssembly version of [Graphviz](https://graphviz.org/) for creating graph visualizations in web browsers and Node.js.
+
+### Key Features
+- **Multiple Layout Engines**: dot, neato, fdp, sfdp, circo, twopi, osage, patchwork
+- **Multiple Output Formats**: SVG, DOT, JSON, Plain text
+- **Image Support**: Can embed external images in graphs
+- **Browser & Node.js Compatible**: Works in both environments
+
+## Package Structure
+
+```
+packages/graphviz/
+├── src/
+│   ├── index.ts          # Package entry point
+│   ├── graphviz.ts       # Main Graphviz class implementation
+│   └── __package__.ts    # Package metadata
+├── src-cpp/              # C++ Graphviz source
+│   ├── main.cpp          # C++ implementation
+│   ├── main.hpp          # C++ headers
+│   └── main.idl          # WebIDL bindings
+├── tests/                # Test files
+│   ├── graphviz.spec.ts  # Main tests
+│   ├── worker.*.spec.ts  # Web Worker tests
+│   └── dot001.js         # Test data
+└── package.json          # Package configuration
+```
+
+## API Overview
+
+### Main Class
+```typescript
+export class Graphviz {
+    static async load(): Promise<Graphviz>
+    
+    // Primary layout method
+    layout(dot: string, format?: Format, engine?: Engine, options?: GraphvizOptions): string
+    
+    // Convenience methods
+    dot(dot: string, options?: GraphvizOptions): string  // SVG output with dot engine
+    circo(dot: string, options?: GraphvizOptions): string
+    fdp(dot: string, options?: GraphvizOptions): string
+    neato(dot: string, options?: GraphvizOptions): string
+    osage(dot: string, options?: GraphvizOptions): string
+    patchwork(dot: string, options?: GraphvizOptions): string
+    sfdp(dot: string, options?: GraphvizOptions): string
+    twopi(dot: string, options?: GraphvizOptions): string
+    
+    // Utility methods
+    version(): string
+}
+```
+
+### Types and Enums
+```typescript
+// Output formats
+type Format = "svg" | "dot" | "json" | "dot_json" | "xdot_json" | "plain" | "plain-ext" | "canon"
+
+// Layout engines  
+type Engine = "circo" | "dot" | "fdp" | "sfdp" | "neato" | "osage" | "patchwork" | "twopi" | "nop" | "nop2"
+
+// Image support for external images
+interface Image {
+    path: string;
+    width: string;
+    height: string;
+}
+
+interface GraphvizOptions {
+    images?: Image[];
+    yInvert?: boolean;
+    nop?: number;
+}
+```
+
+## Usage Patterns
+
+### Basic Usage
+```typescript
+import { Graphviz } from "@hpcc-js/wasm-graphviz";
+
+const graphviz = await Graphviz.load();
+const svg = graphviz.dot('digraph G { Hello -> World }');
+```
+
+### With Images
+```typescript
+const svg = graphviz.layout(
+    'digraph { a[image="https://example.com/image.png"]; }', 
+    "svg", 
+    "dot", 
+    { 
+        images: [{ 
+            path: "https://example.com/image.png", 
+            width: "272px", 
+            height: "92px" 
+        }] 
+    }
+);
+```
+
+### Different Engines
+```typescript
+// Force-directed layouts
+const fdp_svg = graphviz.fdp('graph G { a -- b -- c }');
+const neato_svg = graphviz.neato('graph G { a -- b -- c }');
+
+// Hierarchical layouts  
+const dot_svg = graphviz.dot('digraph G { a -> b -> c }');
+
+// Circular layouts
+const circo_svg = graphviz.circo('graph G { a -- b -- c -- d -- a }');
+```
+
+## Testing Patterns
+
+### Standard Tests
+```typescript
+import { describe, it, expect } from "vitest";
+import { Graphviz } from "@hpcc-js/wasm-graphviz";
+
+describe("Graphviz", () => {
+    it("loads successfully", async () => {
+        const graphviz = await Graphviz.load();
+        expect(graphviz).toBeDefined();
+    });
+
+    it("generates SVG", async () => {
+        const graphviz = await Graphviz.load();
+        const svg = graphviz.dot('digraph G { a -> b }');
+        expect(svg).toContain('<svg');
+        expect(svg).toContain('</svg>');
+    });
+});
+```
+
+### Browser vs Node.js Tests
+- **Browser tests**: Located in `*.browser.spec.ts`
+- **Node.js tests**: Regular `*.spec.ts` files
+- **Worker tests**: Test usage in Web Workers
+
+## Common DOT Language Patterns
+
+### Basic Directed Graph
+```dot
+digraph G {
+    a -> b -> c;
+    a -> d;
+}
+```
+
+### Styled Nodes and Edges
+```dot
+digraph G {
+    node [shape=box, style=filled, color=lightblue];
+    edge [color=red, style=dashed];
+    
+    start -> process -> end;
+}
+```
+
+### Subgraphs/Clusters
+```dot
+digraph G {
+    subgraph cluster_0 {
+        label="Process 1";
+        a -> b;
+    }
+    
+    subgraph cluster_1 {
+        label="Process 2"; 
+        c -> d;
+    }
+    
+    b -> c;
+}
+```
+
+## C++ Integration
+
+### WebIDL Bindings (main.idl)
+```webidl
+interface Graphviz {
+    void Graphviz();
+    DOMString layout([Const] DOMString src, [Const] DOMString format, [Const] DOMString engine);
+    DOMString version();
+};
+```
+
+### C++ Implementation Notes
+- Uses Graphviz C libraries compiled with Emscripten
+- Memory management handled automatically
+- Thread-safe for single-threaded JavaScript environment
+
+## Debugging and Troubleshooting
+
+### Common Issues
+1. **Invalid DOT syntax**: Check DOT language syntax
+2. **Missing images**: Ensure image URLs are accessible and dimensions provided
+3. **Large graphs**: May cause memory issues in browser
+4. **Engine limitations**: Some engines work better with specific graph types
+
+### Debug Output
+```typescript
+// Enable verbose output (if supported)
+const graphviz = await Graphviz.load({ debug: true });
+```
+
+### Performance Considerations
+- Large graphs (>1000 nodes) may be slow
+- SVG output can be large for complex graphs
+- Consider using simpler engines (nop, nop2) for layout-only operations
+
+## AI Assistant Guidelines
+
+### When Modifying This Package
+1. **Understand DOT language**: Familiarize yourself with Graphviz DOT syntax
+2. **Test with multiple engines**: Different engines have different characteristics
+3. **Consider output formats**: SVG for display, JSON for data processing
+4. **Handle errors gracefully**: Invalid DOT should not crash the application
+5. **Memory awareness**: Large graphs can consume significant memory
+
+### Safe Modifications
+- Adding new convenience methods following existing patterns
+- Updating TypeScript types and interfaces
+- Adding new test cases
+- Documentation improvements
+
+### Modifications Requiring C++ Changes
+- Adding new Graphviz features not exposed via current bindings
+- Performance optimizations
+- Memory management improvements
+- Adding new output formats or engines
+
+### Testing Requirements
+- Test with various DOT language constructs
+- Test all supported engines and formats
+- Test image embedding functionality
+- Test error handling with invalid input
+- Test both browser and Node.js environments
\ No newline at end of file

From 2011d7e25cd1f1562abfaddca0a49e9b09568664 Mon Sep 17 00:00:00 2001
From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com>
Date: Fri, 11 Jul 2025 06:21:38 +0000
Subject: [PATCH 3/4] Add copilot summary and complete instructions
 implementation

Co-authored-by: GordonSmith <966863+GordonSmith@users.noreply.github.com>
---
 .copilot-summary.md | 77 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 77 insertions(+)
 create mode 100644 .copilot-summary.md

diff --git a/.copilot-summary.md b/.copilot-summary.md
new file mode 100644
index 00000000..140c5782
--- /dev/null
+++ b/.copilot-summary.md
@@ -0,0 +1,77 @@
+# Copilot Instructions Summary
+
+This directory contains comprehensive instructions for AI assistants working with the HPCC-JS-WASM repository.
+
+## File Overview
+
+### Main Instructions
+- **[.copilot-instructions.md](.copilot-instructions.md)** - Start here! Repository overview, architecture, and basic guidance
+- **[.copilot-workflow.md](.copilot-workflow.md)** - Step-by-step development workflows for common tasks
+- **[.copilot-patterns.md](.copilot-patterns.md)** - Quick reference for package patterns and usage examples
+- **[.copilot-troubleshooting.md](.copilot-troubleshooting.md)** - Debugging guide for common issues
+
+### Package-Specific Instructions
+- **[packages/graphviz/.copilot-package.md](packages/graphviz/.copilot-package.md)** - Graphviz package specifics
+- **[packages/duckdb/.copilot-package.md](packages/duckdb/.copilot-package.md)** - DuckDB package specifics
+
+## Quick Start for AI Assistants
+
+1. **First Time Setup**: Read [.copilot-instructions.md](.copilot-instructions.md) for repository overview
+2. **Development Tasks**: Follow workflows in [.copilot-workflow.md](.copilot-workflow.md)
+3. **Package Usage**: Check patterns in [.copilot-patterns.md](.copilot-patterns.md)
+4. **Problems?**: Consult [.copilot-troubleshooting.md](.copilot-troubleshooting.md)
+
+## Key Points for AI Assistants
+
+- **Monorepo Structure**: 8 packages, each providing WASM versions of C++ libraries
+- **Build Complexity**: Requires C++ compilation for full functionality
+- **TypeScript First**: Many operations can be done with TypeScript-only builds
+- **Test Expectations**: Fresh clone tests will fail without WASM builds (expected)
+- **Safe Operations**: Documentation changes, TypeScript modifications don't need full build
+
+## Most Common Scenarios
+
+### Making TypeScript Changes
+```bash
+npm ci
+npm run lint
+npm run build-ws  # May fail without WASM, but can still work on TS
+```
+
+### Full Development Setup
+```bash
+npm ci
+npm run install-build-deps  # Requires system tools
+npm run build-cpp          # Compiles C++ to WASM
+npm run build-ws           # Builds TypeScript
+npm run test               # Full test suite
+```
+
+### Quick Fix Workflow
+```bash
+# Make changes
+npm run lint
+npm run lint-fix  # Auto-fix linting issues
+# Test specific package if needed
+cd packages/[package] && npm test
+```
+
+## When to Escalate
+
+- C++ compilation errors requiring emscripten expertise
+- System dependency issues (cmake, python, etc.)
+- Platform-specific problems (Windows, specific Linux distributions)
+- Memory/performance issues beyond normal optimization
+
+## Success Metrics
+
+You've successfully used these instructions if:
+- You understand the repository structure and build system
+- You can make TypeScript changes safely
+- You can diagnose and resolve common build/test failures
+- You know when to attempt full builds vs. TypeScript-only changes
+- You can help users with package-specific questions
+
+---
+
+*These instructions are designed to help AI assistants provide better support for the HPCC-JS-WASM repository. They should be updated as the repository evolves.*
\ No newline at end of file

From 6d09dab2e8a90a216c3e4fa614d11311c8de0eea Mon Sep 17 00:00:00 2001
From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com>
Date: Fri, 11 Jul 2025 06:35:04 +0000
Subject: [PATCH 4/4] Move copilot instructions to .github/instructions folder

Co-authored-by: GordonSmith <966863+GordonSmith@users.noreply.github.com>
---
 .../instructions/README.md                    | 22 +++++++++----------
 .../instructions/main.md                      |  0
 .../instructions/patterns.md                  |  0
 .../instructions/troubleshooting.md           |  0
 .../instructions/workflow.md                  |  0
 README.md                                     |  8 +++----
 6 files changed, 15 insertions(+), 15 deletions(-)
 rename .copilot-summary.md => .github/instructions/README.md (65%)
 rename .copilot-instructions.md => .github/instructions/main.md (100%)
 rename .copilot-patterns.md => .github/instructions/patterns.md (100%)
 rename .copilot-troubleshooting.md => .github/instructions/troubleshooting.md (100%)
 rename .copilot-workflow.md => .github/instructions/workflow.md (100%)

diff --git a/.copilot-summary.md b/.github/instructions/README.md
similarity index 65%
rename from .copilot-summary.md
rename to .github/instructions/README.md
index 140c5782..88748e8a 100644
--- a/.copilot-summary.md
+++ b/.github/instructions/README.md
@@ -1,25 +1,25 @@
-# Copilot Instructions Summary
+# Instructions Summary
 
 This directory contains comprehensive instructions for AI assistants working with the HPCC-JS-WASM repository.
 
 ## File Overview
 
 ### Main Instructions
-- **[.copilot-instructions.md](.copilot-instructions.md)** - Start here! Repository overview, architecture, and basic guidance
-- **[.copilot-workflow.md](.copilot-workflow.md)** - Step-by-step development workflows for common tasks
-- **[.copilot-patterns.md](.copilot-patterns.md)** - Quick reference for package patterns and usage examples
-- **[.copilot-troubleshooting.md](.copilot-troubleshooting.md)** - Debugging guide for common issues
+- **[main.md](main.md)** - Start here! Repository overview, architecture, and basic guidance
+- **[workflow.md](workflow.md)** - Step-by-step development workflows for common tasks
+- **[patterns.md](patterns.md)** - Quick reference for package patterns and usage examples
+- **[troubleshooting.md](troubleshooting.md)** - Debugging guide for common issues
 
 ### Package-Specific Instructions
-- **[packages/graphviz/.copilot-package.md](packages/graphviz/.copilot-package.md)** - Graphviz package specifics
-- **[packages/duckdb/.copilot-package.md](packages/duckdb/.copilot-package.md)** - DuckDB package specifics
+- **[../../packages/graphviz/.copilot-package.md](../../packages/graphviz/.copilot-package.md)** - Graphviz package specifics
+- **[../../packages/duckdb/.copilot-package.md](../../packages/duckdb/.copilot-package.md)** - DuckDB package specifics
 
 ## Quick Start for AI Assistants
 
-1. **First Time Setup**: Read [.copilot-instructions.md](.copilot-instructions.md) for repository overview
-2. **Development Tasks**: Follow workflows in [.copilot-workflow.md](.copilot-workflow.md)
-3. **Package Usage**: Check patterns in [.copilot-patterns.md](.copilot-patterns.md)
-4. **Problems?**: Consult [.copilot-troubleshooting.md](.copilot-troubleshooting.md)
+1. **First Time Setup**: Read [main.md](main.md) for repository overview
+2. **Development Tasks**: Follow workflows in [workflow.md](workflow.md)
+3. **Package Usage**: Check patterns in [patterns.md](patterns.md)
+4. **Problems?**: Consult [troubleshooting.md](troubleshooting.md)
 
 ## Key Points for AI Assistants
 
diff --git a/.copilot-instructions.md b/.github/instructions/main.md
similarity index 100%
rename from .copilot-instructions.md
rename to .github/instructions/main.md
diff --git a/.copilot-patterns.md b/.github/instructions/patterns.md
similarity index 100%
rename from .copilot-patterns.md
rename to .github/instructions/patterns.md
diff --git a/.copilot-troubleshooting.md b/.github/instructions/troubleshooting.md
similarity index 100%
rename from .copilot-troubleshooting.md
rename to .github/instructions/troubleshooting.md
diff --git a/.copilot-workflow.md b/.github/instructions/workflow.md
similarity index 100%
rename from .copilot-workflow.md
rename to .github/instructions/workflow.md
diff --git a/README.md b/README.md
index f9274154..8cc1c2c9 100644
--- a/README.md
+++ b/README.md
@@ -30,10 +30,10 @@ Built with:
 
 This repository includes comprehensive instructions for AI assistants and GitHub Copilot:
 
-* [Main Copilot Instructions](.copilot-instructions.md) - Repository overview and architecture
-* [Development Workflow](.copilot-workflow.md) - Step-by-step development processes
-* [Package Patterns](.copilot-patterns.md) - Common patterns and quick reference
-* [Troubleshooting Guide](.copilot-troubleshooting.md) - Debugging and problem resolution
+* [Main Instructions](.github/instructions/main.md) - Repository overview and architecture
+* [Development Workflow](.github/instructions/workflow.md) - Step-by-step development processes
+* [Package Patterns](.github/instructions/patterns.md) - Common patterns and quick reference
+* [Troubleshooting Guide](.github/instructions/troubleshooting.md) - Debugging and problem resolution
 * Package-specific guides in `packages/*/` directories
 
 ## Version 3 Changes