Skip to content

badlydrawnrob/data-playground

Repository files navigation

README

A soft play area for data!

Goals

  1. Very light data analysis
  2. Very small app prototyping
  3. Storing Marketing/Ads data

Examples such as the Programming Flashcards App, Google Ads, Simple Analytics research, Library Data, and so on. Should involve basically ZERO database admin, unless absolutely necessary.

SQLite (and SQL)

An sqlite cheat sheet ...

/* Launch sqlite */
sqlite3 database.sqlite

/* Schema information */
.schema --indent
.schema Table --indent

/* Table information (orm)*/
pragma table_info(Table);
pragma table_xinfo(Table);

/* Column types (real) */
select typeof(Column) from Table
limit 1;

/* Clear lines */
.shell clear

/* Quit sqlite */
.quit
.exit

Handy settings

Save SQLite settings in ~/.sqliterc

-- Show table headings and column format
.headers on
.mode column

Cleanup

To save space remove all unecessary folders once you're done.

# Delete the environment (use `uv sync` to restore)
rm -rf .venv && rm uv.lock

# Check the command before running it (delete all cache folders)
find . -type d -name "__pycache__" | wc -l
find . -type d -name "__pycache__" -exec echo rm -r {} \; # `-f` is dangerous!

# Run the command (add the `-v` flag to show all files deleted)
find . -type d -name "node_modules" -exec echo rm -r {} \;

Tools

True as of 2025. There are lots of frameworks for dealing with data ...

Python isn't the ideal language for performant data transfer, but it's quite easy to get started with. Other languages like Ocaml, Elixir/Go/Gleam/Roc, might be worth a look at some point (other popular languages here). Remember the "sunk cost fallacy" and "opportunity cost" and cognitive load it takes to learn a new thing. Is the performance 10x better? Worthy of investing 3 months of your time?

GUIs

ORMs

There's async and sync and not all ORMs will work for async.

Utilities

These are great for quickly working with your data

Scaling

SQLite is growing in popularity for production servers

Books

  1. SQLite books

Courses

Google "udemy/coursera sql course"

I think the original SQL courses I did ages back are on Udemy.

⚠️ Warnings

Security

  1. Writing raw SQL (things to watch out for)
  2. About ORMS (safer than raw SQL)

SQLite

Types

Sqlite is very permissive. It isn't at all Type safe!

SQLite datatypes are very simple, with main ones being NULL, INTEGER, TEXT (and occasionally BLOB). You can enable strict tables (or strict mode), but some ORMs use custom types so these will restrict you somewhat. By default SQLite will guarantee NOT NULL and UNIQUE values, but you've got to take care not to introduce bugs when INSERTing row values. Postgres is far more strict. An example below:

-- Creates `null` column name (missing name) 
ALTER TABLE BandMember ADD COLUMN TEXT;
-- Not strict by default! 🤦
-- Ignores types and bounds; infers types if incorrect.
CREATE TABLE shit_types (a INT, b VARCHAR(10));
INSERT INTO shit_types (a,b) VALUES('123',1234567891011);
-- 123|1234567891011

Pragmas

You must notify SQLite to respect foreign keys for every connection! This will make sure that DELETE cascades (if foreign key deleted)

-- For every connection, set this ...
-- Especially for `INSERT` and `DELETE`
PRAGMA foreign_keys=on;

About

A soft play area for data!

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published