Refactor rendering crate to clarify primary Processing API

[tychedelia]

This PR makes a significant number of changes to the current project with the goal of pulling apart the different API objects in the core Processing API and implementing the rest of basic rendering functionality. As such, it may be difficult to review as a single unit. Let's discuss these individual changes.

1. Moves implementations from processing_render/src/lib.rs to their own modules. It's nice to keep lib.rs as the entry point to all our functionality, but the implementations should live somewhere else.
2. Refactors implementations to use the "inner system" pattern first established in Initial image implementation #5. This helps avoid borrowck errors and makes it easier to work with pulling stuff out from the Bevy ECS world.
3. Splits apart and clarifies which objects do what:
   • surface no longer owns everything and is demoted to the equivalent of a Bevy RenderTarget.
   • graphics is made explicit. This is a Camera in Bevy and the owner of rendering commands. Implicitly has a relationship to a surface.
4. Adds support for off-screen rendering.
5. Ditches the PImage naming convention. This isn't idiomatic in Rust and is very Java-ish. I went back and forth here. The Rust-y thing would be to call this ProcessingImage, but that's kind of verbose. The even more Rust-y thing would just to be to rely on the module to disambiguate. Our implementers will import processing::Image. The downside here is that many of these names conflict with other stuff. I think that's okay as it's mostly in our lib.
6. Implements manual readback/pixel update. This is a bit of a pain!

[catilac]

So on graphics::readback we queue up a command to copy the texture to the buffer. then we wait for that to complete yes?

and in the example the way you have it, is you're doing the readback after it's done with drawing not while it's drawing? what would happen if you did a readback before graphics_end_draw?

EDIT: Ah. OK. Right. This all goes into bevy renderer world.

[catilac]

as we start to solidify all of this, it could be nice to have a diagram outlining the relationships of our bevy entities

[tychedelia]

yeah, for sure, in ecs terms this is an "archetype" i.e. the table that a given entity lives in and which components it has

[tychedelia]

So on graphics::readback we queue up a command to copy the texture to the buffer. then we wait for that to complete yes?

Yes, basically we store a staging buffer for every texture to make available to copy into and map into CPU address space. This is typically a non-blocking operation and the code here makes it blocking by waiting on a channel.

and in the example the way you have it, is you're doing the readback after it's done with drawing not while it's drawing? what would happen if you did a readback before graphics_end_draw?

Short answer, yes. You can readback whenever. As long as you flush first. Our begin_draw hook doesn't really do anything right now. And our end_draw hook is just a flush plus activating the camera to present to the swapchain. Normally a flush will run the render graph for that camera without actually presenting, accumulating the drawing in the graphics texture as needed. In other words, we need to make writes visible before we can readback.

Long answer: So, deep within the Bevy renderer, there's something called ViewTarget, which is basically an intermediate rendering texture (i.e. not the swapchain) that is owned by a camera. This structure represents actually a few different textures, but mainly a double-buffered sampled rendering attachment and a MSAA resolve target (i.e. unsampled "main" texture), plus a depth attachment for a given camera. In the basic flow of rendering, a render graph will execute a number of passes which bind these attachments, and then at the end of the graph the MSAA resolve target texture will blitted into the swapchain. So in Processing terms, this is our "graphics" texture, whereas the Bevy camera RenderTarget is our "surface" texture.

The camera's ViewTarget double buffers the render attachments so that users can do what's called a post-process write, which basically means that you bind the current full-screen texture to a render pass so you can do some kind of full-screen effect. This is to get around the fact that you cannot use a texture as a render attachment and read from it at the same time. So we have to ping-pong back and forth every time we need to do a full-screen pass.

When users manually update pixels on the CPU, we write those pixels into the main view target texture, i.e. the unsampled MSAA resolve target, in what's effectively a buffer to texture copy. However, this presents a problem, because when MSAA is enabled, this main texture is just the resolve target not the render color attachment. In other words, the actual "source of truth" is the multi-sampled color attachment. But you can't write directly into a sampled texture because it's not just pixels it's sub-samples per logical pixel.

Render engines handle this by doing something called MSAA writeback at the beginning of a new frame/render graph pass, which basically ensures that the current state of the "main" texture, i.e. resolve target, is "written back" into the sampled color attachment. In other words, this is a post-process write where we blit the previous main texture into the new ping-pong'd sampled texture, aka a "reverse resolve" where we expand the single sampled texture (the one we can manually write into) back into sub-samples.

Whew! MSAA makes everything kinda complicated! But the tl;dr here is that "flush" is necessary in order to make sure that things are "settled" - that all our pending draw commands have been issued and that all our manual writes have been materialized such that they're in the right place. This is... somewhat problematic for topline performance! Outside of the blocking nature of readback, it also means that we have to do a lot of work (multiple redundant passes) in order to present a single frame. This is why it's inefficient.

[catilac]

we love this obviously

[catilac]

yay this all works. thanks for answering my million questions