Progress, scroll bars and cake programming!
There's a time in every project I've worked on where the TODO-list grows faster than it shrinks and just getting one step forward seems like a daunting task. Although BEdit is a fairly small project, it is no exception to this rule. The only way I've found to get forward is to simply, "just do it!". Today I noticed, I'm having a hard time to figure out what to put in the TODO-list (albeit, it's already of non-trivial size)!
The last month or so almost all development have been for the GUI and to my surprise, simple features like scroll bars are fairly tricky to get right. If that's just due to me not having much experience in this style of application development or if it's just naturally tricky is a different story.
Consider doing scroll bars the naïve way: draw the entire view, check the lowest pixel drawn and there you have the height of the area. If the area is larger than the clip region, you have all the data required to determine scroll bar size and position. This works very well, you can have dynamic layouts and the scrolling will just work. The problem is that BEdit is a binary file editor, and binary files may be small, large, huge or of ludicrous size and as such drawing the entire view is simply not feasible for some views. Doing a pre-pass to determine line count is for the same reason not feasible. The solution I went with is to determine the line count before doing the layout. See for example the hex view (check screenshots in project page), before rendering the lines I do know the width of the area, I know the width of the digit groups and hence I know the width of a line. A simple divide will say how many lines are required to draw the entire file. This puts a huge strain on what kind of layouts you can do, you must know the amount of lines before you start drawing. This means the structured view, that is a tree-hierarchy, must be done in two passes - one to figure out the amount of lines, one to draw it. But can't we have both methods? Yes, we can have the cake and eat it too (I call this cake programming). This does shove the problem to the usage code by some extent but it doesn't disable a good feature.
I had similar experiences last month regarding data storage of the binary file where you want minimal extra bookkeeping, quick insertions and random access. But with these features implemented, the major parts of the GUI is done and there's a small shine of light coming from the end of the first-release tunnel.
So what has been done and what's left before alpha release?
Done (alpha stage / usable)
Before initial release
After initial release
... and probably much more that I just haven't imagined yet.
The last month or so almost all development have been for the GUI and to my surprise, simple features like scroll bars are fairly tricky to get right. If that's just due to me not having much experience in this style of application development or if it's just naturally tricky is a different story.
Consider doing scroll bars the naïve way: draw the entire view, check the lowest pixel drawn and there you have the height of the area. If the area is larger than the clip region, you have all the data required to determine scroll bar size and position. This works very well, you can have dynamic layouts and the scrolling will just work. The problem is that BEdit is a binary file editor, and binary files may be small, large, huge or of ludicrous size and as such drawing the entire view is simply not feasible for some views. Doing a pre-pass to determine line count is for the same reason not feasible. The solution I went with is to determine the line count before doing the layout. See for example the hex view (check screenshots in project page), before rendering the lines I do know the width of the area, I know the width of the digit groups and hence I know the width of a line. A simple divide will say how many lines are required to draw the entire file. This puts a huge strain on what kind of layouts you can do, you must know the amount of lines before you start drawing. This means the structured view, that is a tree-hierarchy, must be done in two passes - one to figure out the amount of lines, one to draw it. But can't we have both methods? Yes, we can have the cake and eat it too (I call this cake programming). This does shove the problem to the usage code by some extent but it doesn't disable a good feature.
I had similar experiences last month regarding data storage of the binary file where you want minimal extra bookkeeping, quick insertions and random access. But with these features implemented, the major parts of the GUI is done and there's a small shine of light coming from the end of the first-release tunnel.
So what has been done and what's left before alpha release?
Done (alpha stage / usable)
- General layout of GUI, two panes with editors on left and right
- Editable config file, a binary file of course
- Hex editor
- Structured editor, display of members in order of definition
- By address editor, display of members in order of address
- Source code viewer (currently read-only)
- Unsigned binary, octal, decimal and hex member
- String member
- Enum member (flag-like enums not yet supported)
- Assertions
- Prints
- Release platform, itch.io page awaiting release.
Before initial release
- Tax office, determine pricing etc. (don't worry, no in-app purchases!)
- Better arrays / repeating members
- Signed integers
- Float editor
- Flag-like enums
- Handle infinite loops (or close to infinite) in layout instructions
- Support really large binary files (probably through memory mapping)
- Handle edit conflicts, e.g. file has been modified by other program while being loaded in BEdit
- Hot-reload mode of layout source code, to allow you to modify the layout code in external text editor
After initial release
- Text editor for layout code
- Better visuals and more widget types
- Better interactions between editors (click to jump etc.)
- Layout code debugger
- Binary diff view
- RLE hex view (something like 0x00 <repeats 126 times> 0xFA ...)
- Plotting view
- More member types and perhaps custom ones (half floats, color type, ...)
- Mac and Linux support
- End-user feedback and bug fixes
... and probably much more that I just haven't imagined yet.
Why can't you do the layout of huge file ? Is it a memory or a performance issue ? Assuming the binary file doesn't change each frame, could you do the layout once and cache the result ?
Do you mean you've figured out a way to not do two passes ? If so, can you explain ? What is different about the usage code ?
In the past weeks I've been working on UI code myself, trying to figure out a way to do layout of "complex-ish" things (mixing UI controls, text with wrapping, images and adding or removing things during the layout). I couldn't do it without making several passes, due in big part by a self imposed constraint that I don't want a frame of delay when the layout changes to reflect the change in scrollbar size and displayed content.
In my system (which is a week old and so probably has issues), I have 6 layout passes, some can be ran several times, some might be skipped.
I start with _size_1: compute the size and position of the content, if the content fits in the view rectangle, no scrollbar go to _input.
If the content doesn't fit, you need a scrollbar, but since the scrollbar take part of the width, you need to restart the layout. I can early out the first pass as soon as the height is more than the view height (I'm not sure I'm actually doing this at the moment) and start _size_1 again. If you've got a scrollbar, you need to know the height of the content to display the "elevator" with the right size so you need to finish _size_1. Note that if I had scrollbars on the last frame, I will start doing the layout assuming I'll need scrollbars which is likely and avoids doing the first _size_1 most of the time. I then move to _scrollbar.
_scrollbar handle the view scrollbar input (it's a very short pass). If you move the scrollbar, the content that is in the view rectangle changes, and in my case I only generate "layout info" for things that are in the view, so if the view scrolls I also need to redo the layout after. So if the scrollbar was used, I got to _size_2, other wise I go to _input.
Nothing special in _size_2 it just does the layout and moves to _input.
_input: until here, the UI system did nothing when its functions where called. When _input starts, it sets up the UI system to process mouse and keyboards inputs. In practice it's something like this:
If the _input pass modifies the layout (for example checking a checkbox make new buttons appear), the control that modifies the layout needs to set a flag in the layout system to indicates that we need to redo a size pass (_size_3) before drawing. Otherwise we go to _draw.
Nothing special in _size_3 it just does the layout and moves to _draw.
_draw goes over layout items, and if they are in view, draws them.
Kipt
But can't we have both methods? Yes, we can have the cake and eat it too (I call this cake programming). This does shove the problem to the usage code by some extent but it doesn't disable a good feature.
Do you mean you've figured out a way to not do two passes ? If so, can you explain ? What is different about the usage code ?
In the past weeks I've been working on UI code myself, trying to figure out a way to do layout of "complex-ish" things (mixing UI controls, text with wrapping, images and adding or removing things during the layout). I couldn't do it without making several passes, due in big part by a self imposed constraint that I don't want a frame of delay when the layout changes to reflect the change in scrollbar size and displayed content.
In my system (which is a week old and so probably has issues), I have 6 layout passes, some can be ran several times, some might be skipped.
- layout_pass_size_1
- layout_pass_scrollbar
- layout_pass_size_2
- layout_pass_input
- layout_pass_size_3
- layout_pass_draw
I start with _size_1: compute the size and position of the content, if the content fits in the view rectangle, no scrollbar go to _input.
If the content doesn't fit, you need a scrollbar, but since the scrollbar take part of the width, you need to restart the layout. I can early out the first pass as soon as the height is more than the view height (I'm not sure I'm actually doing this at the moment) and start _size_1 again. If you've got a scrollbar, you need to know the height of the content to display the "elevator" with the right size so you need to finish _size_1. Note that if I had scrollbars on the last frame, I will start doing the layout assuming I'll need scrollbars which is likely and avoids doing the first _size_1 most of the time. I then move to _scrollbar.
_scrollbar handle the view scrollbar input (it's a very short pass). If you move the scrollbar, the content that is in the view rectangle changes, and in my case I only generate "layout info" for things that are in the view, so if the view scrolls I also need to redo the layout after. So if the scrollbar was used, I got to _size_2, other wise I go to _input.
Nothing special in _size_2 it just does the layout and moves to _input.
_input: until here, the UI system did nothing when its functions where called. When _input starts, it sets up the UI system to process mouse and keyboards inputs. In practice it's something like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | if ( layout.pass == layout_pass_input ) { ui_set_pass( ui, ui_pass_update ); } else if ( layout.pass == layout_pass_draw ) { ui_set_pass( ui, ui_pass_draw ); } else { ui_set_pass( ui, ui_pass_none ); } ... /* ui_pass_update_draw is the default, and used when no layout is involved. */ b32 ui_button( ui_state_t* ui, ui_id_t id, rect2 rect, data_t label, u32 font_id ) { b32 result = false; if ( ui->pass == ui_pass_update_draw || ui->pass == ui_pass_update ) { result = ui_button_update( ui, id, rect ); } if ( ui->pass == ui_pass_update_draw || ui->pass == ui_pass_draw ) { ui_button_draw( ui, id, rect, label, font_id ); } return result; } |
If the _input pass modifies the layout (for example checking a checkbox make new buttons appear), the control that modifies the layout needs to set a flag in the layout system to indicates that we need to redo a size pass (_size_3) before drawing. Otherwise we go to _draw.
Nothing special in _size_3 it just does the layout and moves to _draw.
_draw goes over layout items, and if they are in view, draws them.
Hmm... I was going to write an answer here but it would be a bit on the lengthy side :D In short I did initially try and wanted to just go through the binary file and do the layout byte-by-byte. I added culling to the renderer so drawing outside the clip region was more or less a no-op but even then; iterating a 0.5 GB binary file was too slow. Caching was something I considered but if loading a 4-5 GB file starts adding seconds to startup time it's not that feasible.
I'll make a new blog entry later this week going through the current UI structure, what's left to implement and explore, as well as the restrictions of the current decisions.
As a funny sidenote, the command line viewer uses mostly double passes to do the layout. Of course, that's just for console output so it's far simpler. It's a bit on the experimental side but you can see an example of it here.
I'll make a new blog entry later this week going through the current UI structure, what's left to implement and explore, as well as the restrictions of the current decisions.
As a funny sidenote, the command line viewer uses mostly double passes to do the layout. Of course, that's just for console output so it's far simpler. It's a bit on the experimental side but you can see an example of it here.