Bitwise Data Engine (BDE)

BDE is an open-core, blazing-fast, bitmap-powered query engine for analytics and filtering, built in Java. It uses RoaringBitmap for ultra-fast set operations and supports a powerful DSL for querying.

Features

• Bitmap index for categorical and numeric data
• Set algebra queries: AND, OR, NOT, IN, BETWEEN, >, <, !=
• Extensible DSL (ANTLR-based) for expressive queries
• CLI runner for interactive or batch queries
• Pluggable persistence (mmap, disk I/O)
• Modular, testable architecture

Project Structure

bde/
├── core/        # Core bitmap engine
├── storage/     # Persistence: mmap, disk I/O
├── api/         # CLI & REST API layer
├── dsl/         # ANTLR grammar & visitor
├── utils/       # Compression, config
├── data/        # Sample datasets
├── tests/       # JUnit tests

Quick Start

1. Install Dependencies

• Java 17+
• RoaringBitmap
• ANTLR4
• JUnit 5 (for tests)

2. Generate ANTLR Parser

cd bde/dsl
antlr4 -Dlanguage=Java BDEQuery.g4 -o generated

3. Build & Run CLI

javac -cp .:../core:../utils:../storage:../dsl:RoaringBitmap.jar:antlr-4.13.1-complete.jar bde/api/BDECLI.java
java -cp .:../core:../utils:../storage:../dsl:RoaringBitmap.jar:antlr-4.13.1-complete.jar bde.api.BDECLI

4. Run Tests

javac -cp .:../core:../utils:../storage:../dsl:RoaringBitmap.jar:junit-platform-console-standalone-1.9.3.jar bde/tests/BitmapIndexTest.java
java -jar junit-platform-console-standalone-1.9.3.jar --class-path . --scan-class-path

Example DSL Queries

FILTER plan=pro & country=IN | COUNT
FILTER plan!=free | COUNT
FILTER plan IN (pro,free) | COUNT
FILTER age BETWEEN 25 AND 40 | COUNT
FILTER (plan=pro & country=IN) | plan=free | COUNT
FILTER NOT plan=free
FILTER age>25 | COUNT
FILTER age<40 | COUNT

License

Open-core, Apache 2.0. Contributions welcome!

---

🚦 How to Use ANTLR Without the antlr4 Script

You can always invoke ANTLR directly using the jar file. Here’s how:

1. Find the ANTLR jar path

For Homebrew, it’s usually:

/opt/homebrew/Cellar/antlr/4.13.2/libexec/antlr-4.13.2-complete.jar

(If you’re on Intel, it would be /usr/local/Cellar/antlr/4.13.2/libexec/antlr-4.13.2-complete.jar.)

You can confirm with:

brew list antlr

---

2. Run ANTLR with Java

From your bde/dsl directory, run:

java -jar /opt/homebrew/Cellar/antlr/4.13.2/libexec/antlr-4.13.2-complete.jar -Dlanguage=Java BDEQuery.g4 -o .

(Adjust the path if your version or location is different!)

---

3. (Optional) Create an Alias for Convenience

Add this to your ~/.zshrc:

alias antlr4='java -jar /opt/homebrew/Cellar/antlr/4.13.2/libexec/antlr-4.13.2-complete.jar'

Then reload your shell:

source ~/.zshrc

Now you can use antlr4 as a command.

---

Try the java -jar ... command above and you should see the generated Java files. Let me know if you hit any issues!

---

🔴 What does this mean?

Maven cannot find the bde.api.BDECLI class. This usually means:

• The source files are not in the correct directory structure under src/main/java/
• Or, the package declaration does not match the directory structure
• Or, the files are not being compiled by Maven because they are not in the standard Maven source tree

---

🟢 How to Fix

1. Check Your Project Structure

For Maven, your Java files should be in:

<code_block_to_apply_changes_from>

NOT in /Users/raghavendramachikatla/dbe/bde/api/BDECLI.java etc.

---

2. How to Move Your Files

If your files are in /Users/raghavendramachikatla/dbe/bde/api/BDECLI.java, move them to the Maven standard location:

mkdir -p /Users/raghavendramachikatla/dbe/src/main/java
mv /Users/raghavendramachikatla/dbe/bde/* /Users/raghavendramachikatla/dbe/src/main/java/

(Repeat for all subfolders: core, api, dsl, storage, utils.)

---

3. Re-run Maven

After moving, from /Users/raghavendramachikatla/dbe:

mvn compile
mvn exec:java -Dexec.mainClass='bde.api.BDECLI'

---

4. Tests

JUnit tests should be in:

/Users/raghavendramachikatla/dbe/src/test/java/bde/tests/BitmapIndexTest.java

Move your test files similarly.

---

📝 Summary

• Move all your Java source files to src/main/java/bde/...
• Move test files to src/test/java/bde/...
• Re-run Maven from the project

---

**If you need, I can re-create the full set of source files for you in the correct Maven structure. Would you