When we first started implementing the code for this website, we had not yet decided on which database technology we wanted to use. We were weighing the pros and cons between using MySQL and MongoDB. In the mean time, however, we still wanted to start implementing the skeleton for our website. Therefore, we needed to use some temporary solution, which would be easy to implement (but not necessarily a good database choice overall), which we would use until we decided upon which database technology to use. We chose to build that solution ourselves: a simple ndjson based database solution which we've published into @byte-this/simple-nd-base.
Simple-Nd-Base
This library provides an abstraction over reading and writing ndjson files. Ndjson is a file format where each line is a separate JSON structure. It is equivalent to storing each line as a separate item in some large array, except the system can read the file line by line and parse each structure separately, rather than needing to read the entire file and parse the entire array at once. From there, the system can perform common database type operations:
- Filter Records: read records based on certain criteria as defined by a callback function.
- Insert Records: append records to the file.
- Update and Delete: update or delete records based on some criteria as defined by a callback function.
We recommend you use this in conjunction with a data access layer in this manner such as this:
- Create interfaces for objects which need to read and write to the database. These objects should only do that, nothing else (single responsibility).
- Implement these classes with an implementation which consumes this file database library.
- When it is time, implement new classes using the database of your choice and use those moving forward.
For more information and a deeper analysis of this methodology, visit our article on the topic linked here.
How to Use
The @byte-this/simple-nd-base project is available as a consumable package and the code is available on Git:
- Npm Package to directly install and use.
- Github Project to view the source code.
To get started now: you can directly install via npm:
Once installed:
- Create connection objects which use the basic CRUD operations provided in this FileDatabase class.
- Have classes which need to read/write to the database consume connection objects from the step above.
For any use case where more than one connection class needs to connect to the same database, use the flyweight pattern to share the FileDatabase connection between these classes. The simplest way would be to declare the FileDatabase as a dependency in the constructor, then new one up for each file and inject accordingly. Otherwise, if there are any duplicate FileDatabase instances for the same physical file, a race condition may occur.
The FileDatabase class implements both interfaces shown below. When creating connection objects, utilize the functionality provided: