P1SCode Toolchain Workflow

This page documents the practical source-to-artifact flow for P1SCode projects.

Primary Build Driver

  • Tool: SDK/p1sdk/p1sc.py

  • Commands:

    • transpile <file.p1s>

    • check <file.p1s>

    • build <file.p1s>

    • package <file.p1s>

check validates import resolution, tokenization, and transpilation without invoking Cargo. The tokenizer labels p1scode_profile.json / default keyword list as KEYWORD tokens; other words are IDENTIFIER. The Python driver uses p1sc_lexer_unified, aligned with the Rust lexer in p1sc_syntax; host tooling can compare streams via dump_tokens (see § Rust parser parity). Add --json-diagnostics to emit diagnostics as a machine-readable JSON array on stdout (see below). Use --stdin-source with --stdin-path <virtual-file.p1s> to validate unsaved buffer content from stdin. If stdin content uses relative imports, add --stdin-import-root <dir> to control import resolution.

Host Rust transpile (p1sc_emit)

The host crate SDK/p1sdk/p1sc_emit merges imports (p1sc_syntax), parses to AST, and emits boilerplate + bodies using the same sys / io / fs / math / str lowerings as p1sc.py (BOILERPLATE_TOP + transpile()).

cargo build --manifest-path SDK/p1sdk/p1sc_emit/Cargo.toml
SDK/p1sdk/p1sc_emit/target/<host-triple>/debug/p1sc_emit transpile path/to/file.p1s [--emit-rust out/main.rs] [--import-root <dir>]
SDK/p1sdk/p1sc_emit/target/<host-triple>/debug/p1sc_emit build path/to/file.p1s [--project-root <repo>] [--out-app <dir>] [--import-root <dir>]
SDK/p1sdk/p1sc_emit/target/<host-triple>/debug/p1sc_emit package path/to/file.p1s [same flags] [--package-name <stem>] [--package-dir <dir>]

build / package mirror p1sc.py: write p1scode_app Cargo.toml + src/main.rs, copy apps/test_app/link.ld, run cargo +nightly rustc --release -Z build-std=core,alloc -Z json-target-spec --target <SDK/toolchain/x86_64-unknown-p1start.json> with -C link-arg=-Tlink.ld, then write p1scode_build_manifest.json. The manifest tool field is p1sc_emit (Python stays p1sc.py) so you can tell which driver produced the artifact.

p1sc.elf on the image stays check-only; p1sc_emit is for host build/automation (Rust-only substitute for the Python transpile + cargo step).

One-shot host checks: run python SDK/p1sdk/host_smoke.py (fixtures + p1sc_syntax / p1sc_emit tests + p1sc_emit build on hello_world.p1s). Use --skip-build if you only want lex/parser/transpile checks without nightly build-std.

JSON diagnostics (p1sc.py --json-diagnostics)

When this flag is set:

  • Human-oriented progress lines from p1sc.py are suppressed; stdout is reserved for a single JSON value on exit.

  • Payload shape: a JSON array of objects. Each object has:

    • code (string): stable identifier, e.g. P1SC-ECLI001. Codes containing -E are errors; others may be treated as warnings by the bridge.

    • message (string): human-readable description.

    • file (string or null): absolute path when the issue is tied to a file.

    • line (integer or null): 1-based source line when applicable.

    • column (integer or null): 1-based column when applicable.

Representative codes (current toolchain):

Code

Meaning

P1SC-ECLI001

Source file not found

P1SC-ECLI002

Project root not discovered or invalid (missing toolchain JSON)

P1SC-ECLI003

Failed reading primary source

P1SC-ECLI004

Missing source path / stdin configuration

P1SC-ECLI005

Failed reading stdin source

P1SC-ECLI006

Unknown CLI argument(s)

P1SC-EIMP001

Unresolved import "…" (file not found)

P1SC-EIMP002

Import file unreadable

P1SC-EIMP003

Malformed import syntax

P1SC-EBUILD001

Linker script missing during emit

P1SC-EBUILD002

cargo build failed

p1sc_diag_bridge.py loads this array, may add a derived severity (error / warning) for --format json, and maps to LSP-shaped items (uri, range, code, message, …) for --format lsp. Use --group-by-uri or --lsp-publish for alternate LSP container layouts.

On-image syntax check (/bin/p1sc.elf)

The ramdisk ships p1sc.elf (apps/p1sc): same lexer/parser/AST and p1sc_syntax::import_resolve merge rules as p1sc.py. File imports are expanded against the on-image filesystem before lex/parse; os/, std/, p1/ imports are stripped (virtual). --import-root <dir> matches Python --stdin-import-root for the entry file’s resolution base. There is no build / transpile / package on device; use check (explicit or implicit).

Invocation (typical):

p1sc check --import-root /path/to/root /path/to/file.p1s
p1sc /path/to/file.p1s          # implicit check
p1sc check --json-diagnostics /path/to/file.p1s
p1sc check --quiet /path/to/file.p1s
p1sc check --verbose /path/to/file.p1s   # lexer sample + AST dump
p1sc check                     # optional: parses built-in sample (not with --json-diagnostics)
p1sc -h

Exit status: 0 if lex+parse succeed, 2 if import resolution / merge fails (P1SC-EIMP*), 1 for other CLI/read/lexer/parser errors.

--json-diagnostics follows the same stdout JSON array shape as p1sc.py (objects with code, message, file, line, column). Human-oriented output is suppressed so stdout stays a single JSON value.

Diagnostic codes: P1SC-ECLI006 matches p1sc.py (unknown argv tokens). P1SC-ELEX001 and P1SC-ESYNTAX001 are native-only (Rust lexer/parser). P1SC-EIMP001, P1SC-EIMP002, P1SC-EIMP003 match the Python driver when import "…" resolution fails. Overlapping read/usage codes: P1SC-ECLI001, P1SC-ECLI004.

Code

Meaning

P1SC-EIMP001

Unresolved import "…"

P1SC-EIMP002

Import file unreadable

P1SC-EIMP003

Malformed import syntax

P1SC-ELEX001

Lexer rejected input (illegal character, unterminated string, …)

P1SC-ESYNTAX001

Parser error

Cargo build errors (P1SC-EBUILD*) remain host p1sc.py / build concerns.

Platform contract: staging paths, session/PTY expectations, and M1 scope — newdocs/p1scode-m1.md in the P1Start repo.

Other JSON artifacts (not diagnostics):

  • apps/p1scode_app/p1scode_build_manifest.json — build metadata after build / package

  • build/p1scode/packages/<name>/package.json — package manifest from package

  • SDK/p1sdk/fixtures/*.json — conformance catalogues consumed by run_fixtures.py

Build Outputs

Default build output location:

  • generated Rust app crate: apps/p1scode_app

  • ELF artifact: apps/p1scode_app/target/x86_64-unknown-p1start/release/p1scode_app

  • build manifest: apps/p1scode_app/p1scode_build_manifest.json

package command additionally emits:

  • package root: build/p1scode/packages/<package-name>/

  • package manifest: package.json

  • packaged ELF copy: <package-name>.elf

Conformance Fixtures

Fixture runner:

  • SDK/p1sdk/run_fixtures.py

Fixture sources and expectations:

  • SDK/p1sdk/fixtures/

  • SDK/p1sdk/fixtures/cases.json for pass cases

  • SDK/p1sdk/fixtures/error_cases.json for expected-failure diagnostics

Use fixtures to validate token/transpile expectations before changing language rules. Pass fixtures cover hello_world, control flow (while, if / else), project-local import (see module_use.p1s), str.concat + sys.print, and math.random. Error fixtures currently cover import resolution, malformed import syntax, and key CLI diagnostics. Error fixtures also validate JSON diagnostics mode for CI/editor integration safety.

Rust parser parity (host)

The on-device p1sc.elf binary shares lexer/parser/AST with the p1sc_syntax library crate (SDK/p1sdk/p1sc_syntax/). Import expansion is implemented in import_resolve (same algorithm as p1sc.py), exercised by host unit tests with a virtual FS and by p1sc.elf via syscalls. parse_program_with_imports merges then parses; dump_tokens can lex merged sources for parity work:

cargo run --manifest-path SDK/p1sdk/p1sc_syntax/Cargo.toml --bin dump_tokens -- --merge-imports path/to/root.p1s
# stdin + merge: add --source-path /virt/path/to/root.p1s [--import-root DIR]

Host unit tests load SDK/p1sdk/fixtures/*.p1s: they accept the same pass fixtures as CI (plus module_math.p1s and bad_import_missing.p1s — syntax-only; missing modules are not parser errors on raw parse), reject bad_import_syntax.p1s at parse time, and fixture_module_use_merged_program_parses asserts merge+parse matches the transpiler’s expanded unit. A separate test pins SDK/p1sdk/p1scode_profile.json keywords to lexer::KEYWORDS (edit both together when adding a keyword).

  • Manual: cargo test --manifest-path SDK/p1sdk/p1sc_syntax/Cargo.toml --target <host-triple>
    (The repo root .cargo/config.toml defaults to UEFI; pass an explicit host triple, e.g. x86_64-pc-windows-msvc or x86_64-unknown-linux-gnu.)

  • CI: run_fixtures.py runs these tests after the Python transpiler and bridge checks.

Starter template (SDK/template/p1scode-app)

  • main.p1s — entry file with import "os/sys" and a sibling module under lib/ (same resolution rules as fixtures).

  • build.ps1 — from repo root:

    • -Check — p1sc check only (fast CI gate, no Cargo).

    • -Check -JsonDiagnostics — same, with --json-diagnostics on stdout (see § JSON diagnostics).

    • -Transpile — p1sc transpile (validate + emit generated app crate; no Cargo).

    • -Build — p1sc build (transpile + cargo release; no package directory).

    • Default — p1sc package into build/p1scode/packages/… (same as before).

Runtime Tooling Bridge

  • apps/p1s_pkg validates package directory manifests (install command).

  • apps/p1s_linker executes built ELF artifacts (run command).

  • SDK/p1sdk/p1sc_diag_bridge.py wraps p1sc.py --json-diagnostics and emits editor-friendly diagnostics (--format json|text|lsp). Unknown bridge flags yield P1SC-ECLI006 in the same shapes as p1sc.py (no subprocess).

  • For --format lsp, add --group-by-uri to emit a uri -> diagnostics[] map for direct publish-style integrations.

  • For --format lsp, add --lsp-publish to emit a publish-ready list of { "uri": ..., "diagnostics": [...] }.

  • Bridge also supports unsaved buffers with --stdin-source and --stdin-path.

  • Bridge forwards --stdin-import-root so stdin buffers can resolve project-relative imports.

These tools are now operational bridge utilities for packaging/execution workflows rather than pure placeholder output.