feat(cli): add offline PDF parsing CLI#4
Open
hzhaoy wants to merge 14 commits into
Open
Conversation
Add complete litepdf offline PDF to Markdown CLI tooling, including: - CLI entrypoint with parse/batch commands and page range support - Local parser using PyMuPDF for text/image extraction - Error handling and JSON output envelopes - Test fixtures and evaluation suites for validation - Proper build configuration and dependency setup
This commit overhauls the CLI implementation: 1. Replaces argparse with Typer for better CLI UX and consistent error handling 2. Adds a proper custom PageSpecError type instead of using argparse ArgumentTypeError 3. Updates error code documentation and usage error messaging 4. Refactors the page range parsing validation to happen at the CLI layer 5. Maintains all original functionality while improving maintainability and error reporting
Unify the command surface around pdf-parser so parse and batch share the offline Markdown/envelope contract with serve in one Typer app. The old text/json parse path and litepdf console script are removed to avoid two meanings for parse, while the shared runner/output modules keep the parsing contract reusable. The bundled skill is renamed to pdf-parser so usage guidance matches the installed command. Constraint: parse and batch must preserve the existing offline Markdown, manifest, profiling, JSON envelope, and NDJSON batch behavior Rejected: Keep litepdf as a compatibility alias | user confirmed no scripts depend on it Confidence: high Scope-risk: moderate Reversibility: clean Directive: Do not reintroduce a second console script or parse contract without checking README, pyproject entry points, and skill trigger metadata Tested: uv sync; uv run pdf-parser --help; uv run python -m pdf_parser --help; uv run pdf-parser parse skills/pdf-parser/evals/fixtures/normal.pdf --out /tmp/pdf-parser-skill-rename-smoke; uv run ruff check .; uv run ruff format --check .; uv run mypy pdf_parser; uv run --with pyyaml python /Users/bytedance/.codex/skills/.system/skill-creator/scripts/quick_validate.py skills/pdf-parser; git diff --check Not-tested: Full HTTP server runtime beyond serve help output
This commit fully renames the project from pdf-parser to hi-pdf-parser: - Updates all CLI command references, entry points, and package paths - Moves all source code from pdf_parser/ to hi_pdf_parser/ - Updates pyproject.toml build configs and dependencies - Updates pre-commit and CI pipeline paths - Moves and updates skill manifests, eval fixtures and configs - Removes old pdf-parser specific files and cleans up legacy code - Fixes uv lockfile package source type
hzhaoy
changed the title
Move logging setup out of the CLI namespace so command handlers can capture parser and app module logs through the shared hi_pdf_parser package logger. The server module no longer configures root logging at import time, leaving output routing to the entrypoint or host server. Constraint: CLI stdout must remain reserved for JSON envelopes. Rejected: Keep handlers on hi_pdf_parser.cli | parser module logs would still bypass per-document stderr.log. Confidence: high Scope-risk: narrow Directive: Keep library modules emitting normal module loggers; configure handlers only from entrypoints. Tested: uv run python -m unittest tests.test_logging_setup Tested: uv run ruff check hi_pdf_parser tests Tested: uv run ruff format --check hi_pdf_parser tests Tested: uv run mypy hi_pdf_parser tests Tested: hi-pdf-parser parse fixture preserves JSON stdout and captures parser logs in stderr.log Tested: hi-pdf-parser serve health endpoint returned 200
ealyn
reviewed
Jul 3, 2026
Centralize PyMuPDFParser creation behind a package-level factory so parse, batch, and serve do not maintain separate default config construction paths. Constraint: Runtime options remain at each caller; HTTP request options and CLI page ranges are still passed to parser.parse per invocation. Rejected: Move request/runtime options into the factory | that would make per-call behavior look like instance defaults and risk changing HTTP semantics. Confidence: high Scope-risk: narrow Directive: Keep create_parser limited to parser instance defaults; do not route command output or HTTP response behavior through it. Tested: uv run python -m unittest tests.test_logging_setup tests.test_parser_factory Tested: uv run ruff check hi_pdf_parser tests Tested: uv run ruff format --check hi_pdf_parser tests Tested: uv run mypy hi_pdf_parser tests Tested: hi-pdf-parser parse fixture returned success and wrote document.md Tested: hi-pdf-parser serve health endpoint returned 200
Rename parse and batch support modules out of hi_pdf_parser.cli so the package has one command surface instead of a CLI-only support namespace. The moved modules now describe their command, artifact, envelope, and page-range responsibilities directly. Constraint: Preserve parse, batch, and serve command behavior while removing the cli package boundary. Rejected: Route serve through parse_runner | serve returns HTTP responses while parse_runner owns file artifacts, envelopes, and exit codes. Confidence: high Scope-risk: narrow Directive: Keep output artifact/envelope code separate from HTTP response code; share only lower-level parser construction and parse options. Tested: uv run python -m unittest tests.test_logging_setup tests.test_parser_factory Tested: uv run ruff check hi_pdf_parser tests Tested: uv run ruff format --check hi_pdf_parser tests Tested: uv run mypy hi_pdf_parser tests Tested: hi-pdf-parser parse fixture returned success and wrote document.md Tested: hi-pdf-parser batch fixtures returned two success envelopes Tested: hi-pdf-parser serve health endpoint returned 200
Remove the parse_runner module and keep parse/batch execution in the unified command entrypoint. The batch command now owns its aggregation loop directly, while parse and batch share a private single-file helper for the artifact/envelope workflow. Constraint: Preserve stdout envelope contracts, per-document artifacts, exit codes, and serve behavior. Rejected: Duplicate the full single-file parse flow inside both parse and batch | that would make error mapping and artifact writes drift. Confidence: high Scope-risk: moderate Directive: Keep shared parse/batch file handling as private helpers unless another non-command surface needs the artifact/envelope contract. Tested: uv run python -m unittest tests.test_commands tests.test_logging_setup tests.test_parser_factory Tested: uv run ruff check hi_pdf_parser tests Tested: uv run ruff format --check hi_pdf_parser tests Tested: uv run mypy hi_pdf_parser tests Tested: hi-pdf-parser parse fixture returned success and wrote document.md Tested: hi-pdf-parser batch partial failure returned exit code 1 with success,error NDJSON Tested: hi-pdf-parser serve health endpoint returned 200
Introduce ParseRuntimeOptions as the single per-call option object for parser.parse kwargs. CLI page ranges and HTTP request options now share the same conversion path while parser config defaults remain in parser_factory. Constraint: Runtime options must stay per invocation; do not mix password, extract flags, or page ranges into parser instance defaults. Rejected: Pass None-valued options through to parser.parse | omitting unspecified values preserves config fallback semantics and avoids noisy kwargs. Confidence: high Scope-risk: narrow Directive: Add new per-call parser kwargs to ParseRuntimeOptions before wiring them through CLI or HTTP handlers. Tested: uv run python -m unittest tests.test_app tests.test_commands tests.test_logging_setup tests.test_parser_factory tests.test_parse_runtime Tested: uv run ruff check hi_pdf_parser tests Tested: uv run ruff format --check hi_pdf_parser tests Tested: uv run mypy hi_pdf_parser tests Tested: hi-pdf-parser parse multipage fixture with --pages 1 reported one page Tested: hi-pdf-parser batch partial failure returned exit code 1 with success,error NDJSON Tested: hi-pdf-parser serve health endpoint returned 200
Inline the local success path into the single-file parse helper and rename the parser call helper to describe its returned blocks and metadata. This removes one command-layer hop without changing artifact, envelope, or error mapping behavior. Constraint: Keep parser/PyMuPDF exception translation separate from artifact/envelope orchestration. Rejected: Inline parser exception mapping into _parse_file | it would mix PyMuPDF details with command output handling. Confidence: high Scope-risk: narrow Directive: Keep _parse_pdf_blocks focused on parser invocation and exception translation only. Tested: uv run python -m unittest tests.test_app tests.test_commands tests.test_logging_setup tests.test_parser_factory tests.test_parse_runtime Tested: uv run ruff check hi_pdf_parser tests Tested: uv run ruff format --check hi_pdf_parser tests Tested: uv run mypy hi_pdf_parser tests Tested: hi-pdf-parser parse multipage fixture with --pages 1 reported one page Tested: hi-pdf-parser batch partial failure returned exit code 1 with success,error NDJSON Tested: hi-pdf-parser serve health endpoint returned 200
Keep the Typer entrypoint focused on command wiring while moving local parse execution into command_runner and artifact materialization into artifact_writer. Thin wrappers for page ranges, parser construction, and per-call parse options are removed because their behavior is now covered by direct call paths and tests. Constraint: Do not reintroduce a cli support package or keep thin forwarding modules Rejected: Keep ParseRuntimeOptions | it only wrapped direct kwargs after request parameters were reduced Rejected: Keep parser_factory.py | construction helpers only forward into PyMuPDFParser Confidence: high Scope-risk: moderate Directive: Keep parser-instance defaults in PyMuPDFParserConfig and pass per-call options directly Tested: uv run python -m unittest discover -s tests Tested: uv run ruff check --config=pyproject.toml hi_pdf_parser tests Tested: uv run --no-sync mypy hi_pdf_parser Tested: uv run ruff format --check --config=pyproject.toml hi_pdf_parser tests Tested: git diff --check
Merge the latest origin/main release-prep changes into feat/cli while preserving the refactored parse and batch command boundaries. The conflict resolution keeps the command_runner path for offline parsing and adopts the optional server-extra import behavior for serve.
Constraint: origin/main moved FastAPI and uvicorn behind the server extra
Rejected: Restore the old direct parser CLI implementation | it would undo the command_runner boundary cleanup
Confidence: high
Scope-risk: moderate
Directive: Keep serve-only imports inside serve so CLI parse/batch work without server dependencies
Tested: uv lock --check
Tested: uv run --extra server python -m unittest discover -s tests
Tested: uv run ruff check --config=pyproject.toml hi_pdf_parser tests
Tested: uv run --extra server mypy hi_pdf_parser
Tested: uv run ruff format --check --config=pyproject.toml hi_pdf_parser tests
Tested: git diff --cached --check
Tested: uv run --no-sync python -c "import hi_pdf_parser.__main__; print('cli-import-ok')"
ealyn
reviewed
Jul 7, 2026
ealyn
reviewed
Jul 7, 2026
| help="PDF Parser CLI and server.", | ||
| epilog="""示例: | ||
| hi-pdf-parser parse report.pdf --out ./out | ||
| hi-pdf-parser batch a.pdf b.pdf --out ./out |
Author
There was a problem hiding this comment.
跟计算没关系，主要是在处理多个文件时 agent 只需要一次工具调用就可以
Separate the skill package artifacts from the CLI parser PR while keeping only the fixtures needed by command tests under tests/fixtures. Convert the CLI-facing help and error text to English so the command surface is consistent for reviewers and users.
Constraint: Review requested moving skill-related content to a separate PR
Constraint: Review requested English-only CLI text
Rejected: Remove batch | reviewer concern was discussed as tool-call ergonomics rather than compute savings
Confidence: high
Scope-risk: moderate
Directive: Keep skill packaging out of this PR; add it back in a separate skill/evals branch
Tested: uv run --extra server python -m unittest discover -s tests
Tested: uv run ruff check --config=pyproject.toml hi_pdf_parser tests
Tested: uv run --extra server mypy hi_pdf_parser
Tested: uv run ruff format --check --config=pyproject.toml hi_pdf_parser tests
Tested: git diff --cached --check
Tested: rg -n '[\p{Han}]' hi_pdf_parser tests README.md pyproject.toml
Tested: rg -n 'skills/|hi-pdf-parser/evals|evals/fixtures' .
Tested: uv run --no-sync hi-pdf-parser --help
Tested: uv run --no-sync hi-pdf-parser parse tests/fixtures/multipage.pdf --pages 1,3
Consolidate multi-file handling under parse, remove obsolete mode metadata, and give the output helpers concise responsibility-based module names. Constraint: Keep the CLI surface limited to parse and serve without a cli package. Rejected: Preserve batch and legacy options as aliases | the approved interface intentionally removes them. Rejected: Add absence assertions for removed mode fields | explicitly excluded from the requested test scope. Confidence: high Scope-risk: moderate Directive: Keep parser.py independent of CLI artifact and envelope concerns. Tested: unittest discover (15 tests); Ruff check; Ruff format check; mypy; CLI help smoke; git diff check Not-tested: Downstream consumers of the removed batch command and envelope fields
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
pdf_parser/cliTest