Skip to content

Releases: yellowfeather/DbfDataReader

Release list

v2.2.0

v2.2.0 Latest
Latest

Choose a tag to compare

View all tags
@chrisrichards chrisrichards released this 06 Jul 18:38
v2.2.0
e5fda7f
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

A performance release: reading is cheaper everywhere, and queries stop paying for columns they never touch.

Halved read-path allocations (#303)

Reading every field of every row now allocates half of what it did, reaching allocation parity with the fastest .NET DBF readers:

  • Character fields trim their padding before decoding — one right-sized string per field instead of a full-width string plus a trimmed copy (single-byte and UTF-8 encodings; others keep the decode-then-trim path).
  • Numeric and date fields parse directly from the record buffer via stack-allocated spans — no intermediate ASCII string per field.
  • IsDBNull no longer boxes the value it inspects, and the typed getters (GetDecimal, GetInt32, …) read through the typed value objects instead of boxing and unboxing. A fully typed consumer reads rows with zero allocations beyond the parsed strings themselves.

On the 587-row × 16-column TIGER benchmark fixture the standard read-everything loop went from 675 KB to 327 KB allocated per pass.

Queries parse only the columns they need (#304)

The SQL engine, DbfQuery<T> and COUNT(*) now parse rows in two phases: the columns the WHERE clause references for every candidate row, and the remaining projected (plus, when sorting, ORDER BY) columns only once a row matches. Selective queries over wide tables skip most parsing work — including memo-file reads for unreferenced memo columns:

select NAME from places.dbf where AWATER > 100000000   -- non-matching rows parse AWATER only

Full-scan query benchmarks on a 50,000-row table dropped 17–21% in time and 72–86% in allocations; a selective scan projecting one of sixteen columns runs 3.6× faster with 11× fewer allocations. Index seeks and status scans are unchanged. Values the query never parsed are guarded — accessing one throws a clear InvalidOperationException instead of exposing stale data.

Compatibility notes

  • IDbfValue gains an IsNull property. Types deriving DbfValue<T> inherit it automatically; only direct IDbfValue implementations need to add the member.
  • GetBoolean on a NULL cell now throws SqlNullValueException, consistent with every other typed getter (previously NullReferenceException).
  • Date fields parse with the invariant culture (the format is fixed ASCII digits; the current culture's calendar no longer has any influence).

Full changelog: v2.1.0...v2.2.0

Assets 2
Loading

v2.1.0

v2.1.0

Choose a tag to compare

View all tags
@chrisrichards chrisrichards released this 06 Jul 15:26
v2.1.0
df15938
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Index acceleration grows in three directions: numeric and date keys, COUNT(*), and descending order.

Integer, numeric, double and date index keys (#298)

Non-character index tags now work as automatic access paths — in a typical Visual FoxPro database that unlocks the integer primary/foreign-key tags:

select CALL_ID from calls.dbf where CONTACT_ID = 1          -- index seek (=) on tag 'CONTACT_ID'
select CALL_ID from calls.dbf where CALL_ID between 4 and 9 -- index range scan (between)

Integer keys (4-byte big-endian, sign bit flipped) are verified byte-for-byte against real VFP files; numeric, double and date keys use the IEEE 754 total-order transform (dates as Julian day numbers). Integer bounds are adjusted to the integer domain (CALL_ID = 1.5 is provably empty without touching the table), and double/date strict bounds stay inclusive at the key level with the residual filter restoring exactness. DateTime and currency keys remain unsupported pending verified fixtures.

COUNT(*) with fast paths (#299)

SELECT COUNT(*) [WHERE …] executes without materializing rows, choosing the cheapest safe strategy — and ExplainPlan() reports which one ran:

  • no WHERE: a record status scan that skips value parsing entirely (header record counts are never trusted);
  • an index that covers the WHERE exactly: the count comes from the index alone, with zero table reads — or with per-record status checks when deleted rows are skipped;
  • anything else: rows are read and filtered but never projected.

COUNT is not a reserved word (columns named count keep working), the result is an int column named count, and ExecuteScalar / Dapper's ExecuteScalar<int> compose naturally. DbfQuery<T>.Count() uses the same fast paths.

Descending ORDER BY via indexes (#300)

order by col desc no longer forces an in-memory sort when the column has an ascending tag: the entry list is reversed, with duplicate-key runs re-stabilized so the result is identical to a stable descending sort. Applies to order-only scans, filtered queries, TOP/LIMIT, and DbfQuery<T>.OrderByDescending. Tags created DESCENDING in VFP remain unsupported pending a fixture (#301 explains how to contribute one).

Full changelog: v2.0.0...v2.1.0

Loading

v2.0.0

v2.0.0

Choose a tag to compare

View all tags
@chrisrichards chrisrichards released this 06 Jul 13:47
v2.0.0
d008142
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

The biggest DbfDataReader release yet: SQL queries, typed results, automatic index use, true async I/O, and random record access.

Highlights

SQL queries over DBF files

The ADO.NET provider now executes real SQL — column projection with aliases, WHERE (comparisons, BETWEEN, IN, LIKE, IS [NOT] NULL, AND/OR/NOT), ORDER BY (multi-key, ASC/DESC), TOP/LIMIT, named (@name) and positional (?) parameters, and ExecuteScalar:

select top 5 Point_ID as id, Date_Visit
from dbase_03.dbf
where Max_PDOP >= 3.5 and Point_ID like 'A%'
order by Max_PDOP desc

String comparisons are ordinal, case-sensitive and ignore trailing spaces (matching DBF collation); NULL follows SQL three-valued logic; date columns compare against 'yyyy-MM-dd' strings. Parse and binding errors carry character positions.

Typed queries

Map rows straight to your own types — either with the LINQ-style builder on DbfTable, whose lambdas are translated into the query engine (the type's properties define which columns are read), or Dapper-style over SQL text on DbfDbConnection:

var points = dbfTable.Query<GpsPoint>()
    .Where(p => p.Max_PDOP >= 3.5m && p.Point_ID.StartsWith("A"))
    .OrderByDescending(p => p.Date_Visit)
    .Take(10)
    .ToList();

var rows = dbConnection.Query<GpsPoint>(
    "select Point_ID, Max_PDOP, Date_Visit from dbase_03.dbf where Point_ID = @id",
    new { id = "A1" });

The provider is also verified against real Dapper, if you prefer it.

Automatic index use

When a compound index (file.cdx) sits next to a table, equality, range, BETWEEN and prefix-LIKE predicates become index seeks, and a matching ORDER BY reads in index order instead of sorting — for SQL text and the typed builder alike. The planner is conservative (unsupported shapes fall back to a full scan, and the full WHERE is always re-applied), UseIndexes=false / .WithoutIndexes() force scans, and ExplainPlan() shows the chosen path.

Visual FoxPro compound index (.cdx) reading

A new DbfDataReader.Cdx namespace opens .cdx files, enumerates their tags, walks entries in key order and searches by key — with CdxKeyEntry.RecordIndex feeding straight into Seek. Ported from the dev/async fork by Dai Rees (@daiplusplus), whose reverse-engineering of the compound tag directory, interior node layout and bit-packed leaf key compression made this possible — thank you!

Async reading

DbfDataReader.ReadAsync is now real async I/O (one buffered read per record feeding the existing span-based parsing), with ValueTask-based DbfTable.ReadAsync/ReadRecordAsync underneath and cancellation support throughout.

Random access

DbfTable.Seek(recordIndex) / DbfDataReader.Seek(recordIndex) jump to any record by zero-based index, and RecordIndex reports the index of the record just read.

Breaking changes

  • Currency ('Y') columns are now decimal-backed (previously float). Use GetDecimal()/GetValue<decimal>() instead of GetFloat(). This matches the advertised column schema — GetDecimal() previously threw — and fixes precision loss on large amounts.
  • The DbfTable stream constructors gained optional parameters (leaveOpen); source-compatible, but assemblies compiled against 1.x should be recompiled.
  • ReadFloatsAsDecimals is now honoured when reading through DbfDataReader and the ADO.NET provider (it was silently ignored before).
  • On netstandard2.1, truncated header/column reads now throw EndOfStreamException instead of silently parsing a half-filled buffer, matching net10.0 behaviour.

Fixes

  • Partial stream reads are handled when reading headers and columns (network/decompression streams).
  • Caller-supplied streams can be kept open with the new leaveOpen constructor parameter.
  • Invalid SQL command text still throws ArgumentException (as SqlParseException, now with error positions).

Full changelog: v1.1.0...v2.0.0

Loading

v1.1.0

v1.1.0

Choose a tag to compare

View all tags
@chrisrichards chrisrichards released this 04 Mar 20:07
v1.1.0
41ded65
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What's Changed

  • Validate day and month in UpdatedAt assignment to prevent invalid dates by @chrisrichards in #260
  • Add parameterless constructor to DbfDbConnection and update tests by @chrisrichards in #261
  • Enhance file path handling in DbfDbCommand and add tests for filename variations by @chrisrichards in #264
  • Refactor encoding handling in DbfDbConnection and DbfDbConnectionStri… by @chrisrichards in #263
  • Bump Microsoft.NET.Test.Sdk from 18.0.1 to 18.3.0 by @dependabot[bot] in #259

Full Changelog: v1.0.0...v1.1.0

Contributors

chrisrichards and dependabot
Loading

v1.0.0

v1.0.0

Choose a tag to compare

View all tags
@chrisrichards chrisrichards released this 23 Feb 20:21
v1.0.0
07a6cdb

What's Changed

Package updates

New Contributors

Full Changelog: v0.9.0...v1.0.0

Contributors

chrisrichards, jasolko, and 7 other contributors
Loading

v0.9.0

v0.9.0

Choose a tag to compare

View all tags
@chrisrichards chrisrichards released this 12 Jun 21:39
v0.9.0
b06bbeb

What's Changed

New Contributors

Full Changelog: v0.8.0...v0.9.0

Contributors

kstrauss, xyrus02, and 3 other contributors
Loading

v0.8.0

v0.8.0

Choose a tag to compare

View all tags
@chrisrichards chrisrichards released this 05 Jul 11:43
v0.8.0
e02dcc1
  • fixes reading of cells with null char value #61
  • fixes handle null/empty string from BinaryReader.ReadString() #20 , #112
  • fixes reading of negative currency values #62
  • fixes reading of number columns with length > 10 #33 #98
  • fixes reading character columns with length > 255 #67
  • sets the encoding from the language driver in the DBF file #131
  • targets net5.0
  • bumps dependencies
Loading

v0.7.0

v0.7.0

Choose a tag to compare

View all tags
@chrisrichards chrisrichards released this 19 Feb 21:26
v0.7.0
84e382d
  • adds net48 target framework
Loading

v0.6.0

v0.6.0

Choose a tag to compare

View all tags
@chrisrichards chrisrichards released this 19 Feb 17:52
v0.6.0
156bf24
  • drops net461 target framework
  • updates target framework to netstandard2.1
  • uses ReadOnlySpan
Loading

v0.5.8

v0.5.8

Choose a tag to compare

View all tags
@chrisrichards chrisrichards released this 18 Feb 22:21
v0.5.8
3676fd3
  • adds implemention of GetSchemaTable to DbfDataReader
Loading