Skip to content

Added integration testings - #30

Closed
Ataba29 wants to merge 10 commits into
mainfrom
test/integration
Closed

Added integration testings#30
Ataba29 wants to merge 10 commits into
mainfrom
test/integration

Conversation

@Ataba29

@Ataba29 Ataba29 commented Jul 26, 2026

Copy link
Copy Markdown
Owner

Summary

This pull request was created in order to make sure that the project is running fully using integration testing and not just unit testing as it is usefull but not enough

Related issue

Closes # NONE

Type of change

  • Bug fix
  • New feature
  • Refactor / internal change
  • Documentation
  • Build / CI

How was this tested?

The project is being tested locally so far after this addition it will add CI testing

Checklist

  • Code builds locally (cmake --build build)
  • Tests pass (ctest --test-dir build)
  • Added/updated tests for the change
  • Updated documentation if needed
  • Follows the project's header-guard / .h-.cpp split / Doxygen-comment conventions

@Ataba29
Ataba29 requested a review from razimograbi as a code owner July 26, 2026 19:12
@github-actions github-actions Bot added the ci label Jul 26, 2026
@Ataba29

Ataba29 commented Jul 26, 2026

Copy link
Copy Markdown
Owner Author

Important note: to make the integration testing failed i found a bug in the connections map, which we fixed in the previous PR, but since the PR was not merged, the new tests are not gonna pass. This will be drafted until all PRs will be added here is the full bug i found:

Bug: Data race on Server::connections between accept and event-loop threads

connections is a plain std::unordered_map accessed from two different threads with no synchronization.

acceptClients() runs on its own thread and writes to it on every new connection:

connections[AcceptSocket] = Connection{AcceptSocket, active_key};

runEventLoop() runs on a separate thread and reads from it (via find) every time an epoll/IOCP event fires:

auto it = connections.find(entry.socket);

closeConnection() also erases from it, and can be invoked from the event loop thread.

None of these three access points hold a lock. std::unordered_map gives no thread-safety guarantee for concurrent reads and writes — a write on one thread and a read on another at the same time is undefined behavior, not just a stale-read risk. Specifically, an insert that triggers a rehash can invalidate the internal bucket structure mid-iteration on the other thread.

Under normal load this is rare because connections arrive spaced out. Under a burst of near-simultaneous connections, the accept thread inserts multiple entries in quick succession while the event loop thread is concurrently reading — increasing the odds of hitting the race window. This shows up in integration testing as sporadic connection resets or refusals under concurrent client load, and in the worst case could corrupt the map or crash the process.

Fix

Guard connections with a std::mutex, held for the insert in acceptClients(), the find/copy in runEventLoop(), and the find/erase in closeConnection() — same pattern already used for busySockets.

@Ataba29
Ataba29 marked this pull request as draft July 26, 2026 19:15
@Ataba29 Ataba29 added bug Something isn't working enhancement New feature or request Test labels Jul 26, 2026
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Jul 26, 2026
@github-actions github-actions Bot added the build label Jul 26, 2026
@Ataba29

Ataba29 commented Jul 26, 2026

Copy link
Copy Markdown
Owner Author

@razimograbi, the new integration tests that I ran found the 2 bugs related to #20 and #21. I have added a fix for #20, but for #21, the fix is in this PR #24. Please review it and approve it so I can merge it and merge this PR as well; these two bugs are pretty nasty; they crash the server.

@razimograbi

Copy link
Copy Markdown
Collaborator

I have a suggestion, instead of adding a guard connections with a std::mutex to the conection. We can make the connections std::unordered_map value a shared_ptr (Which has a reference count. When that reference count drops to 0 we free it.).
I have tried this mechanism inside the Fix Fragmentation pull request and it works, (Also checked if the de-structor is called and indeed it is called.)

Comment thread src/Server/Server.cpp
Comment thread src/Server/Server.h
@razimograbi

Copy link
Copy Markdown
Collaborator

Can you make sure that after the Integration testing the Data is not persisted anywhere. because if we ran the code locally and the data got persisted this will cause problems in terms of testing itself, because we want to operate on clean data.

@razimograbi

Copy link
Copy Markdown
Collaborator

Fire Brother!

last_error = None
for _ in range(self.max_retries):
try:
self.sock.sendall((command + "\n").encode())

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our fragmentation indicator checks if the last byte sent is (0x0A) Ascii code 10 which is "`n" .
You can check here.

You can also check out this

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Client already appends \n on send and waits for it on recv, matches the server's framing check. so no change needed

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What i mean is that \n asccii is not 0x0A .
Do you get what I mean?
if you click enter on you keyboard it doesn't add a \n it will add 0x0A. Lets just align our expectations.
Do we want a simple enter or an intentional /n.

Also I think it should be a '/n' in order for it to be 0x0A.
Not "/n" can you double check?

Comment thread src/Tests/integration/README.md
@Ataba29 Ataba29 mentioned this pull request Jul 27, 2026
@Ataba29

Ataba29 commented Jul 27, 2026

Copy link
Copy Markdown
Owner Author

I will have to add a simple file deletion after running tests to wipe out test data from the log files

@Ataba29

Ataba29 commented Jul 27, 2026

Copy link
Copy Markdown
Owner Author

im stuck in a weird spot the code passes tests on windows but not on docker
need to investigate further, just a note without rate limiting the code runs on docker with a lot of clients

@Ataba29
Ataba29 marked this pull request as ready for review July 27, 2026 16:26
@Ataba29

Ataba29 commented Jul 27, 2026

Copy link
Copy Markdown
Owner Author

@razimograbi Finally got everything to work, had to adjust tests to sit into 1 file because it was running the tests in the wrong order, and I have to add a line that will add the thread lib into windows build test for some reason the last test didnt find it so I manually added it, Added the #24 to this PR and fixed connections thing so code ready for review

@Ataba29 Ataba29 closed this Jul 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working build ci documentation Improvements or additions to documentation enhancement New feature or request Test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants