Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 37 additions & 19 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ uv sync --group dev
```bash
uv run pytest # run all tests
uv run pytest tests/path/to/test_file.py::test_name # run a single test
uv run pytest --cov gems --cov-report xml # with coverage
uv run pytest --cov gems_craft --cov gems_runner --cov gems_craft_hybrid --cov-report xml # with coverage
```

**Lint & Format:**
Expand All @@ -30,26 +30,21 @@ uv run mypy
**Running:**
```bash
# CLI entry point
gemspy \
--model-libs path/to/lib1.yml path/to/lib2.yml \
--components path/to/components.yml \
--timeseries path/to/timeseries/ \
--duration 8760 \
--scenarios 1
gemspy --study path/to/study_dir

# Python API — directory-based study
from gems.study.folder import load_study
from gems.study.runner import run_study
from gems_runner.study.folder import load_study
from gems_runner.study.runner import run_study

study = load_study(Path("path/to/study_dir")) # reads input/, model-libraries/, data-series/
run_study(Path("path/to/study_dir")) # loads study, solves, writes CSV to output/

# Python API — programmatic study
from gems.study import Study
from gems.simulation import build_problem, TimeBlock
from gems_runner.study import Study
from gems_runner.simulation import build_problem, TimeBlock

study = Study(system=system, database=database)
problem = build_problem(study, TimeBlock(1, list(range(8760))), scenarios=1)
problem = build_problem(study, TimeBlock(1, list(range(8760))), scenario_ids=[0])
problem.solve(solver_name="highs")
```

Expand All @@ -59,13 +54,41 @@ The pipeline flows: **YAML input → parsing → model resolution → system ins

An optional `optim-config.yml` activates decomposition: variables and constraints are split across a master problem and subproblems, with either sequential resolution or full Benders decomposition.

### Core Modules (`src/gems/`)
### Core Modules

The source tree is split into three packages under `src/`:

#### `gems_craft/` — I/O and data structures (no solver dependency)

**`model/`** — Immutable model templates.
- `Model`: defines component behavior (parameters, variables, constraints, ports)
- `Library`: a collection of models, loaded from YAML
- `Taxonomy` (`taxonomy.py`): categories naming the items a model must expose. Models opt in via `taxonomy-category`; `check_library_against_taxonomy` enforces conformance.

**`study/`** — Study schema and parsing.
- `SystemSchema` (`parsing.py`): Pydantic model for `system.yml`
- `ScenarioBuilder` (`scenario_builder.py`): maps MC scenarios to data-series column indices;

**`optim_config/`** — Optional decomposition configuration.
- `OptimConfig` (`parsing.py`): top-level config loaded from `optim-config.yml`
- `ResolutionMode` (`parsing.py`): `FRONTAL` (default), `SEQUENTIAL_SUBPROBLEMS`, `PARALLEL_SUBPROBLEMS`, or `BENDERS_DECOMPOSITION`
- `ModelDecompositionConfig` (`parsing.py`): per-model assignment of variables/constraints/objective contributions to master or subproblems

#### `gems_craft_hybrid/` — Hybrid study I/O (read/write only, no simulation)

Extends `gems_craft` schemas for hybrid GEMS studies. **Hybrid studies cannot be simulated with GemsPy** — simulation support is not yet implemented.

**`study/`** — `HybridSystemSchema(SystemSchema)`: adds `area-connections` and `thermal-capacity-connections`.
- Use `load_yaml_system(path, HybridSystemSchema)` and `write_yaml_system(system, path)` from `gems_craft`

**`model/`** — `HybridPortTypeSchema(PortTypeSchema)`: adds `area-connection` and `thermal-capacity-connection`. `HybridLibrarySchema(LibrarySchema)`: overrides `port-types` with `HybridPortTypeSchema`.
- Use `load_yaml_library(path, HybridLibrarySchema)` and `write_yaml_library(library, path)` from `gems_craft`

#### `gems_runner/` — Solver and execution (depends on `gems_craft`)

**`model/`** — Resolves YAML schemas into runtime objects.
- `resolve_library`: validates and cross-links models across libraries

**`expression/`** — Mathematical expression language and AST.
- `ExpressionNode`: base frozen dataclass for all expression tree nodes
- Grammar is defined in `grammar/Expr.g4` and parsed via ANTLR4 (generated files live in `expression/parsing/antlr/` — do not edit directly)
Expand All @@ -74,7 +97,7 @@ An optional `optim-config.yml` activates decomposition: variables and constraint
**`study/`** — Study definition and instantiation.
- `System` (`system.py`): resolved topology — graph of `Component`s, `PortRef`s, and `PortsConnection`s after library references are substituted
- `Study` (`study.py`): dataclass pairing a `System` with a `DataBase`; validates that the database supplies every parameter required by the system
- `DataBase` (`data.py`): manages time-series and scenario data;
- `DataBase` (`data.py`): manages time-series and scenario data
- `load_study` / `run_study` (`folder.py`): convenience functions for directory-based studies (`input/system.yml`, `input/model-libraries/`, `input/data-series/`)

**`simulation/`** — Optimization problem construction and solving.
Expand All @@ -85,11 +108,6 @@ An optional `optim-config.yml` activates decomposition: variables and constraint
- `TimeBlock` (`time_block.py`): defines the temporal window for one solve
- `SimulationTableBuilder` / `SimulationTableWriter` (`simulation_table.py`): result extraction as a flat pandas `DataFrame`

**`optim_config/`** — Optional decomposition configuration.
- `OptimConfig` (`parsing.py`): top-level config loaded from `optim-config.yml`
- `ResolutionMode` (`parsing.py`): `FRONTAL` (default), `SEQUENTIAL_SUBPROBLEMS`, `PARALLEL_SUBPROBLEMS`, or `BENDERS_DECOMPOSITION`
- `ModelDecompositionConfig` (`parsing.py`): per-model assignment of variables/constraints/objective contributions to master or subproblems

**`libs/`** — Resolves the path to bundled YAML model libraries shipped with the package.

### Key Design Patterns
Expand Down
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,12 +50,12 @@ Given a study directory containing your `library.yml`, `system.yml` and timeseri

```python
from pathlib import Path
from gems.study.folder import load_study
from gems.session import SimulationSession
from gems.optim_config import load_optim_config
from gems_runner.study.folder import load_study
from gems_runner.session import SimulationSession
from gems_craft.optim_config import load_yaml_optim_config

study = load_study(Path("my_study"))
optim_config = load_optim_config(Path("my_study/input/optim-config.yml"))
optim_config = load_yaml_optim_config(Path("my_study/input/optim-config.yml"))

session = SimulationSession(study=study, optim_config=optim_config)
results = session.run()
Expand All @@ -65,7 +65,7 @@ Or, in a single call:

```python
from pathlib import Path
from gems.study.runner import run_study
from gems_runner.study.runner import run_study

run_study(Path("my_study"))
```
Expand Down
25 changes: 25 additions & 0 deletions docs/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,31 @@

All notable changes to GemsPy are documented here.

## [2.0.0] - Unreleased

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe rather 1.0.0 or 0.2.0 ?


### Changed

- **Package split**: the `gems` package is replaced by three dedicated packages:
- `gems_craft`: I/O and data structures (YAML parsing/writing for libraries, systems, optim-config, scenario builder) — no solver dependency
- `gems_runner`: solver and execution logic (expression language, model resolution, simulation, session)
- `gems_craft_hybrid`: read/write support for hybrid GEMS studies (not simulable with GemsPy — use Antares Simulator)
- **Renamed functions**: all public YAML functions now follow `load_yaml_*` / `write_yaml_*` with `Path` inputs. Stream-based helpers are private (`_parse_yaml_*`).
- `parse_yaml_library` → `load_yaml_library`
- `parse_yaml_components` / `load_input_system` → `load_yaml_system`
- `load_optim_config` → `load_yaml_optim_config`, `write_optim_config` → `write_yaml_optim_config`
- `ScenarioBuilder.load` → `ScenarioBuilder.load_dat`, `ScenarioBuilder.dump` → `ScenarioBuilder.write_dat`
- **Generic schema support**: `load_yaml_system` and `load_yaml_library` accept an optional `schema` argument (TypeVar with default) to load subclass schemas without code duplication.

### Added

- **`gems_craft_hybrid`** package with:
- `HybridSystemSchema`: extends `SystemSchema` with `area-connections` and `thermal-capacity-connections`
- `HybridPortTypeSchema`: extends `PortTypeSchema` with `area-connection` and `thermal-capacity-connection`
- `HybridLibrarySchema`: extends `LibrarySchema` with `HybridPortTypeSchema` port-types
- **`version`** field on `LibrarySchema`

---

## [0.1.2] - 2026-06-11

### Added
Expand Down
7 changes: 4 additions & 3 deletions docs/agents/python-convention.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,9 @@
is run with `disallow_untyped_defs = true` and `disallow_untyped_calls = true` (see `mypy.ini`).
- **Dataclasses**: Prefer `@dataclass(frozen=True)` for value objects (expression nodes, model
definitions). Mutability must be justified explicitly.
- **Pydantic**: Use `ConfigDict(alias_generator=to_camel)` or kebab-case alias generation for YAML
round-tripping; use Pydantic v2 APIs only.
- **Pydantic**: Use `ModifiedBaseModel` (from `gems_craft.utils`) which applies kebab-case alias
generation via `alias_generator=_to_kebab` and `extra="forbid"`. Use Pydantic v2 APIs only.
Serialize with `model_dump(by_alias=True, exclude_none=True, mode="json")` for YAML output.
- **Naming**:
- Classes: `PascalCase`
- Functions / variables: `snake_case` — use descriptive names; avoid single-letter or two-letter
Expand All @@ -28,7 +29,7 @@
Rules for automated agents (CI bots, AI coding assistants, Dependabot, etc.):

1. **Never auto-merge** to `main`; all changes require at least one human review.
2. **Do not edit generated files** under `src/gems/expression/parsing/antlr/`; regenerate them
2. **Do not edit generated files** under `src/gems_runner/expression/parsing/antlr/`; regenerate them
from `grammar/Expr.g4` instead.
3. **Do not modify `pyproject.toml` version** manually; version bumps are handled via the
`feat(release):` commit workflow.
Expand Down
2 changes: 2 additions & 0 deletions docs/agents/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ in `tests/` sub-directories. No mocking of the solver—tests use real HiGHS cal
| Unit — libraries | `tests/unittests/lib_parsing/` | Model library YAML parsing |
| Unit — system | `tests/unittests/system/` | Model, network, and port object behaviour |
| Unit — system parsing | `tests/unittests/system_parsing/` | System YAML parsing |
| Unit — gems_craft | `tests/unittests/gems_craft/` | Read/write roundtrips for system, library, optim-config, scenario builder |
| Unit — gems_craft_hybrid | `tests/unittests/gems_craft_hybrid/` | Hybrid system and library parsing (HybridSystemSchema, HybridLibrarySchema) |
| Unit — scenario builder | `tests/unittests/scenario_builder/` | Scenario and time-series builder |
| Integration | `tests/unittests/simulation/` | Full problem build + solve on small networks |
| End-to-end — functional | `tests/e2e/functional/` | Cross-cutting tests: library/system combinations, stochastic, investment, scenario builder |
Expand Down
33 changes: 14 additions & 19 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,11 +86,9 @@ The second category of input files mentioned above corresponds to system files.
~~~yaml
system:
model-libraries: basic
nodes:
components:
- id: N
model: basic.node

components:
- id: G1
model: basic.generator
properties:
Expand Down Expand Up @@ -168,12 +166,12 @@ If your inputs are organised in a study directory (see [Reading input files](use

~~~ python
from pathlib import Path
from gems.study.folder import load_study
from gems.session import SimulationSession
from gems.optim_config import load_optim_config
from gems_runner.study.folder import load_study
from gems_runner.session import SimulationSession
from gems_craft.optim_config import load_yaml_optim_config

study = load_study(Path("my_study"))
optim_config = load_optim_config(Path("my_study/input/optim-config.yml"))
optim_config = load_yaml_optim_config(Path("my_study/input/optim-config.yml"))

session = SimulationSession(study=study, optim_config=optim_config)
results = session.run()
Expand All @@ -183,7 +181,7 @@ Or, in a single call:

~~~ python
from pathlib import Path
from gems.study.runner import run_study
from gems_runner.study.runner import run_study

run_study(Path("my_study"))
~~~
Expand All @@ -198,16 +196,13 @@ Here is the GemsPy syntax to read a test case described by

~~~ python
from pathlib import Path
from gems.model.parsing import parse_yaml_library
from gems.model.resolve_library import resolve_library
from gems.study.parsing import parse_yaml_components
from gems.study.resolve_components import resolve_system, build_data_base

with open("library.yml") as lib_file:
input_libraries = [parse_yaml_library(lib_file)]
from gems_craft.model.parsing import load_yaml_library
from gems_runner.model.resolve_library import resolve_library
from gems_craft.study.parsing import load_yaml_system
from gems_runner.study.resolve_components import resolve_system, build_data_base

with open("system.yml") as compo_file:
input_system = parse_yaml_components(compo_file)
input_libraries = [load_yaml_library(Path("library.yml"))]
input_system = load_yaml_system(Path("system.yml"))

result_lib = resolve_library(input_libraries)
system = resolve_system(input_system, result_lib)
Expand All @@ -217,8 +212,8 @@ database = build_data_base(input_system, Path(series_dir))
### Building the optimisation problem

~~~ python
from gems.study import Study
from gems.simulation import build_problem, TimeBlock
from gems_runner.study import Study
from gems_runner.simulation import build_problem, TimeBlock

problem = build_problem(
Study(system, database),
Expand Down
19 changes: 15 additions & 4 deletions docs/user-guide/building.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ The Pydantic schema classes used to describe systems programmatically follow the
The syntax to build components with the GemsPy API is the following:

~~~ python
from gems.study.parsing import ComponentSchema, ComponentParameterSchema
from gems_craft.study.parsing import ComponentSchema, ComponentParameterSchema

components = []

Expand Down Expand Up @@ -76,7 +76,7 @@ components.append(
The syntax to build connections between components with the GemsPy API is the following:

~~~ python
from gems.study.parsing import PortConnectionsSchema
from gems_craft.study.parsing import PortConnectionsSchema

connections = []

Expand All @@ -102,12 +102,23 @@ connections.append(
## Defining a SystemSchema

~~~ python
from gems.study.parsing import SystemSchema
from gems_craft.study.parsing import SystemSchema

input_system = SystemSchema(
components=components,
connections=connections,
)
~~~

The `input_system` variable can then be used in the same way as when it was created using the [parse_yaml_components](inputs.md) method.
The `input_system` variable can then be used in the same way as when it was created using the [load_yaml_system](inputs.md) function.

## Writing a SystemSchema to file

Once you have built or modified a `SystemSchema`, you can save it as a `system.yml` file:

~~~ python
from pathlib import Path
from gems_craft.study.parsing import write_yaml_system

write_yaml_system(input_system, Path("my_study/input/system.yml"))
~~~
Loading
Loading