← Back to index

Tutorial HHA version 0 - Part 2 A Dynamic Language

Jens
#23136

This is part 2 of specifying HHA file format in BEdit layout language.

To recap, we left off having the following output:

>bedit.exe -layout hha.bet -data v0_hhas\intro_art.hha
hha_file
hha_header (.Header)
    0h            MagicValue                                     "hhaf"
   04h               Version                                          0
   08h              TagCount                                        109
   0Ch        AssetTypeCount                                         21
   10h            AssetCount                                         55
   14h                  Tags                                        2Ch
   1Ch            AssetTypes                                      3 94h
   24h                Assets                                      4 90h

hha_tag (.Tags[0])
   2Ch                    ID                             Tag_Smoothness
   30h                 Value                                   0.000000

...

hha_asset_type (.AssetTypes[0])
03 94h                TypeID                                 Asset_None
03 98h       FirstAssetIndex                                          0
03 9Ch OnePastLastAssetIndex                                          0

...

hha_asset (.Assets[0])
04 90h            DataOffset                                         0h
04 98h         FirstTagIndex                                          0
04 9Ch   OnePastLastTagIndex                                          0
04 A0h            DataHeader 0x0000000000000000000000000000000000000000

hha_asset (.Assets[1])
04 B4h            DataOffset                                      C 4Ch
04 BCh         FirstTagIndex                                          1
04 C0h   OnePastLastTagIndex                                          3
04 C4h            DataHeader 0x29090000800400000000003F0000003F00000000

and we would like to have the DataHeader displayed as it's proper type rather than just arbitrary bytes.

The type in question is

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
struct hha_asset
{
    u(8, hex) DataOffset;
    u32 FirstTagIndex;
    u32 OnePastLastTagIndex;
    /* TODO(Jens): We'll soon see how to get this behavior back.
    union
    {
        hha_bitmap Bitmap; // sizeof(hha_bitmap) == 4*4
        hha_sound Sound; // sizeof(hha_sound) == 3*4
        hha_font Font; // sizeof(hha_font) == 5*4
    }; // sizeof union is 20 bytes
    */
    raw(20) DataHeader;
};

but what member of the union should we display and where? The type does not have enough data to determine what's the "active" union member.

Overview of HHA version 0 file format

The data regarding what asset type an hha_asset is, is split in two; the AssetTypes and Assets.

AssetTypes is a list of types containing the range of indices where the actual asset exists.

1
2
3
4
5
6
struct hha_asset_type
{
    asset_type_id(4) TypeID;
    u32 FirstAssetIndex; // <- start of range
    u32 OnePastLastAssetIndex; // <- end of range
};

The TypeID then determines if the hha_asset should be sound, font or bitmap. If we do a quick inspection of the file content we already print out, the 20th AssetType entry looks like

hha_asset_type (.AssetTypes[19])
04 78h                TypeID                      Asset_OpeningCutscene
04 7Ch       FirstAssetIndex                                          1
04 80h OnePastLastAssetIndex                                         55

and tells us that Assets[1..54] has TypeID Asset_OpeningCutscene. If I'm not mistaken, an actual type is determined like

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
if (Asset_Shadow <= type && type <= Asset_Torso || type == Asset_OpeningCutscene || type == Asset_FontGlyph)
{
    hha_bitmap Bitmap;
}
else if (Asset_Bloop <= type && type <= Asset_Puhp)
{
    hha_sound Sound;
}
else if (type == Asset_Font)
{
    hha_font Font;
}

This information needs to be passed to hha_asset.

Passing type parameters

BEdit structs should be seen more like functions than C-style structs. They can read data, have temporary variables and take parameters. Let's start by passing asset type to hha_asset and add if-branches to determine what type we should display.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
struct hha_asset(var type)
{
    u(8, hex) DataOffset;
    u32 FirstTagIndex;
    u32 OnePastLastTagIndex;

    if (Asset_Shadow <= type && type <= Asset_Torso || type == Asset_OpeningCutscene || type == Asset_FontGlyph)
    {
        hha_bitmap Bitmap; // sizeof(hha_bitmap) == 4*4
    }
    else if (Asset_Bloop <= type && type <= Asset_Puhp)
    {
        hha_sound Sound; // sizeof(hha_sound) == 3*4
    }
    else if (type == Asset_Font)
    {
        hha_font Font;  // sizeof(hha_font) == 5*4
    }
    else
    {
        raw(20) DataHeader;
    }
};


struct hha_file
{
    hha_header Header;

    @(Header.Tags) hha_tag Tags[Header.TagCount];
    @(Header.AssetTypes) hha_asset_type AssetTypes[Header.AssetTypeCount];
    @(Header.Assets) hha_asset(Asset_OpeningCutscene /*TODO(Jens): What should the type be here? */) Assets[Header.AssetCount];
};

Let's run again and see the results.

>bedit.exe -layout hha.bet -data v0_hhas\intro_art.hha
188:    if (Asset_Shadow <= type && type <= Asset_Torso || type == Asset_OpeningCutscene || type == Asset_FontGlyph)
            ~~~~~~~~~~~~
ERROR Expected member of type hha_asset, local variable, struct parameter, enumName.value or anonymous enum value.

C-enum to BEdit or BEdit-enum to C

BEdit enum values are by default accessed with enumName.value. We can change Asset_Shadow to asset_type_id.Asset_Shadow but that would be quite verbose. Let's instead change the enum to behave like in C, this is done with anonymous keyword.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
enum anonymous asset_font_type
{
    ...
};

enum anonymous asset_tag_id
{
    ...
};

enum anonymous asset_type_id
{
    ...
};

enum anonymous hha_sound_chain
{
    ...
};

Note that if you don't specify a name for the enum it's anonymous, but then you can't use it for specifying scalar members.

Let's run again and we see

>bedit.exe -layout hha.bet -data v0_hhas\intro_art.hha

...

hha_asset (.Assets[0])
04 90h            DataOffset                       0h
04 98h         FirstTagIndex                        0
04 9Ch   OnePastLastTagIndex                        0
hha_bitmap (.Bitmap)
04 A0h                Dim[0]                        0
04 A4h                Dim[1]                        0
04 A8h    AlignPercentage[0]                 0.000000
04 ACh    AlignPercentage[1]                 0.000000


hha_asset (.Assets[1])
04 B0h            DataOffset        C 4C 00 00 00 00h
04 B8h         FirstTagIndex                        0
04 BCh   OnePastLastTagIndex                        1
hha_bitmap (.Bitmap)
04 C0h                Dim[0]                        3
04 C4h                Dim[1]                    2 345
04 C8h    AlignPercentage[0]                 0.000000
04 CCh    AlignPercentage[1]                 0.500000


hha_asset (.Assets[2])
04 D0h            DataOffset             3F 00 00 00h
04 D8h         FirstTagIndex               10 808 908
04 DCh   OnePastLastTagIndex                        0
hha_bitmap (.Bitmap)
04 E0h                Dim[0]                        3
04 E4h                Dim[1]                        5
04 E8h    AlignPercentage[0]                 0.000000
04 ECh    AlignPercentage[1]                 0.000000

...

that everything is broken. Why? The first (Assets[0]) looks properly zero initialized (HHA has first array entries as dummy values initialized to zero). If we see the difference in address for DataOffset, 04 90h and 04 B0h BEdit thinks the size is 20h (32 in decimal), however the size of hha_asset should be 36.

Adding "union" padding

We changed

1
2
3
4
5
6
union
{
    hha_bitmap Bitmap; // sizeof(hha_bitmap) == 4*4
    hha_sound Sound; // sizeof(hha_sound) == 3*4
    hha_font Font; // sizeof(hha_font) == 5*4
};

to

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
if (Asset_Shadow <= type && type <= Asset_Torso || type == Asset_OpeningCutscene || type == Asset_FontGlyph)
{
    hha_bitmap Bitmap; // sizeof(hha_bitmap) == 4*4
}
else if (Asset_Bloop <= type && type <= Asset_Puhp)
{
    hha_sound Sound; // sizeof(hha_sound) == 3*4
}
else if (type == Asset_Font)
{
    hha_font Font; // sizeof(hha_font) == 5*4
}
else
{
    raw(20) DataHeader;
}

BEdit determines the size of the type dynamically, if the 1st branch is hit the size (of the replaced union) is 16, not 20 as we wanted. To deal with this we have to add padding. If you don't want the padding displayed you can declare the type as hidden.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
struct hha_asset(var type)
{
    u(8, hex) DataOffset;
    u32 FirstTagIndex;
    u32 OnePastLastTagIndex;

    if (Asset_Shadow <= type && type <= Asset_Torso || type == Asset_OpeningCutscene || type == Asset_FontGlyph)
    {
        hha_bitmap Bitmap; // sizeof(hha_bitmap) == 4*4
        hidden(4) Padding;
    }
    else if (Asset_Bloop <= type && type <= Asset_Puhp)
    {
        hha_sound Sound; // sizeof(hha_sound) == 3*4
        hidden(2 * 4) Padding;
    }
    else if (type == Asset_Font)
    {
        hha_font Font; // sizeof(hha_font) == 5*4
    }
    else
    {
        raw(20) DataHeader;
    }
};

If we run again we start getting proper results

>bedit.exe -layout hha.bet -data v0_hhas\intro_art.hha

...

hha_asset (.Assets[1])
04 B4h            DataOffset                 C 4Ch
04 BCh         FirstTagIndex                     1
04 C0h   OnePastLastTagIndex                     3
hha_bitmap (.Bitmap)
04 C4h                Dim[0]                 2 345
04 C8h                Dim[1]                 1 152
04 CCh    AlignPercentage[0]              0.500000
04 D0h    AlignPercentage[1]              0.500000


hha_asset (.Assets[2])
04 D8h            DataOffset             A4 EE 4Ch
04 E0h         FirstTagIndex                     3
04 E4h   OnePastLastTagIndex                     5
hha_bitmap (.Bitmap)
04 E8h                Dim[0]                 2 345
04 ECh                Dim[1]                 1 119
04 F0h    AlignPercentage[0]              0.500000
04 F4h    AlignPercentage[1]              0.500000

If you have a bug, work to not get it again

As we have a strict size requirement on the size of hha_asset let's assert that it's what we expect. BEdit layout language has no sizeof operator but we can still calculate the size by using current_address(). This will return you a copy of the starting address of the next member, since the start of the first member, minus the start of the next member after all members have been specified equal the size of the struct we can do the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
enum
{
    hha_asset_size = 36
};

struct hha_asset(var type)
{
    var startOfType = current_address();

    u(8, hex) DataOffset;
    u32 FirstTagIndex;
    u32 OnePastLastTagIndex;

    if (Asset_Shadow <= type && type <= Asset_Torso || type == Asset_OpeningCutscene || type == Asset_FontGlyph)
    {
        hha_bitmap Bitmap; // sizeof(hha_bitmap) == 4*4
        hidden(4) Padding;
    }
    else if (Asset_Bloop <= type && type <= Asset_Puhp)
    {
        hha_sound Sound; // sizeof(hha_sound) == 3*4
        hidden(2 * 4) Padding;
    }
    else if (type == Asset_Font)
    {
        hha_font Font; // sizeof(hha_sound) == 5*4
    }
    else
    {
        assert(type == 0); // NOTE(Jens): Use zero when you don't know what type it is.
        raw(20) DataHeader;
    }

    var endOfType = current_address();
    assert(endOfType - startOfType == hha_asset_size);
};

Now we're ready to pass in the struct parameter to hha_asset.

Loops

At this time there are two ways we can pass the required information to display an hha_asset. Either we give the starting address (Header.Assets) to hha_asset_type and have that type display the hha_asset, or we have the hha_file iterate through each AssetTypes and pass it to hha_asset from there. Either one will do, and it's up to your own personal preference on what you'd like to have displayed. I opted for the second version, iterate.

Note that, if you opt for the first version you might need to use an external member, those members are just like all other members but they don't increment the current_address. More on this approach on a later entry.

BEdit currently (version 0.0.2) only has support for while loops, but this is enough for us at this time (although a for-loop would've been more intuitive).

EDIT Version 0.0.3 adds support for for-loops as well as do while.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
struct hha_file
{
    hha_header Header;

    @(Header.Tags) hha_tag Tags[Header.TagCount];
    @(Header.AssetTypes) hha_asset_type AssetTypes[Header.AssetTypeCount];

    // NOTE(Jens): We could keep this member but since we're displaying this in another 
    //             way it's just cluttering up the output at this time, in my opinion.
    // @(Header.Assets) hha_asset(Asset_None) Assets[Header.AssetCount];

    var assetTypeIndex = 0;
    while (assetTypeIndex < Header.AssetTypeCount)
    {
        var assetType = AssetTypes[assetTypeIndex].TypeID;

        if (assetType)
        {
            var startIndex = AssetTypes[assetTypeIndex].FirstAssetIndex;
            var count = AssetTypes[assetTypeIndex].OnePastLastAssetIndex - AssetTypes[assetTypeIndex].FirstAssetIndex;

            @(Header.Assets + hha_asset_size * startIndex) hha_asset(assetType) Assets[count];
        }

        assetTypeIndex = assetTypeIndex + 1;
    }
};

Now the output looks like:

>bedit.exe -layout hha.bet -data v0_hhas\intro_art.hha

...

hha_asset (.Assets[0])
04 B4h            DataOffset                 C 4Ch
04 BCh         FirstTagIndex                     1
04 C0h   OnePastLastTagIndex                     3
hha_bitmap (.Bitmap)
04 C4h                Dim[0]                 2 345
04 C8h                Dim[1]                 1 152
04 CCh    AlignPercentage[0]              0.500000
04 D0h    AlignPercentage[1]              0.500000


hha_asset (.Assets[1])
04 D8h            DataOffset             A4 EE 4Ch
04 E0h         FirstTagIndex                     3
04 E4h   OnePastLastTagIndex                     5
hha_bitmap (.Bitmap)
04 E8h                Dim[0]                 2 345
04 ECh                Dim[1]                 1 119
04 F0h    AlignPercentage[0]              0.500000
04 F4h    AlignPercentage[1]              0.500000

...

One important note, we're now starting to deviate from what the file contains to what we want to have displayed. For example we've completely discarded the data given by the first entry of the Assets array. This can be good and bad; good because we don't want a bunch of non-important data displayed and bad because we want to know if we're having bugs in our file format. The BEdit GUI shows both the Hex-editor as well as the formatted (editable) output to help you get the best of both worlds, however at this time it's not ready to be released for a wider audience.

Now we could go on and add the actual pixel data, but as BEdit isn't an image viewer it doesn't bring us much benefit unless we're dealing with structured art (see test.bmp and bmp.bet in the examples folder for an example). But we could extend our display to include the tags.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
enum
{
    hha_tag_size = 8
};

struct hha_tag
{
    asset_tag_id(4) ID;
    r32 Value;
};

...

struct hha_file
{
    hha_header Header;

    @(Header.Tags) hha_tag Tags[Header.TagCount];
    @(Header.AssetTypes) hha_asset_type AssetTypes[Header.AssetTypeCount];

    // NOTE(Jens): This member is being displayed differently, we could keep it but since we're displaying 
    //             this in another way it's just cluttering up the output at this time.
    // @(Header.Assets) hha_asset(Asset_None) Assets[Header.AssetCount];

    var assetTypeIndex = 0;
    while (assetTypeIndex < Header.AssetTypeCount)
    {
        var assetType = AssetTypes[assetTypeIndex].TypeID;

        if (assetType)
        {
            var firstAssetIndex = AssetTypes[assetTypeIndex].FirstAssetIndex;
            var onePastLastAssetIndex = AssetTypes[assetTypeIndex].OnePastLastAssetIndex;

            var assetIndex = firstAssetIndex;
            while (assetIndex < onePastLastAssetIndex)
            {
                @(Header.Assets + hha_asset_size * assetIndex) hha_asset(assetType) Asset;

                var firstTagIndex = Asset.FirstTagIndex;
                var onePastLastTagIndex = Asset.OnePastLastTagIndex;

                @(Header.Tags + hha_tag_size * firstTagIndex) hha_tag AssetTags[onePastLastTagIndex - firstTagIndex];

                assetIndex = assetIndex + 1;
            }
        }

        assetTypeIndex = assetTypeIndex + 1;
    }
};

And as per usual, the output:

>bedit.exe -layout hha.bet -data v0_hhas\intro_art.hha

...

hha_asset (.Asset)
04 B4h            DataOffset                 C 4Ch
04 BCh         FirstTagIndex                     1
04 C0h   OnePastLastTagIndex                     3
hha_bitmap (.Bitmap)
04 C4h                Dim[0]                 2 345
04 C8h                Dim[1]                 1 152
04 CCh    AlignPercentage[0]              0.500000
04 D0h    AlignPercentage[1]              0.500000


hha_tag (.AssetTags[0])
   34h                    ID         Tag_ShotIndex
   38h                 Value              1.000000

hha_tag (.AssetTags[1])
   3Ch                    ID        Tag_LayerIndex
   40h                 Value              1.000000

hha_asset (.Asset)
04 D8h            DataOffset             A4 EE 4Ch
04 E0h         FirstTagIndex                     3
04 E4h   OnePastLastTagIndex                     5
hha_bitmap (.Bitmap)
04 E8h                Dim[0]                 2 345
04 ECh                Dim[1]                 1 119
04 F0h    AlignPercentage[0]              0.500000
04 F4h    AlignPercentage[1]              0.500000


hha_tag (.AssetTags[0])
   44h                    ID         Tag_ShotIndex
   48h                 Value              1.000000

hha_tag (.AssetTags[1])
   4Ch                    ID        Tag_LayerIndex
   50h                 Value              2.000000

...

As the viewer adds one newline when a struct ends, and since the end of hha_asset ends two structs (the hha_bitmap and hha_asset) at the same time, I'd like a separator there. We can add one with print.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
struct hha_file
{
    hha_header Header;

    @(Header.Tags) hha_tag Tags[Header.TagCount];
    @(Header.AssetTypes) hha_asset_type AssetTypes[Header.AssetTypeCount];

    // NOTE(Jens): This member is being displayed differently, we could keep it but since we're displaying 
    //             this in another way it's just cluttering up the output at this time.
    // @(Header.Assets) hha_asset(Asset_None) Assets[Header.AssetCount];

    var assetTypeIndex = 0;
    while (assetTypeIndex < Header.AssetTypeCount)
    {
        var assetType = AssetTypes[assetTypeIndex].TypeID;

        if (assetType)
        {
            var firstAssetIndex = AssetTypes[assetTypeIndex].FirstAssetIndex;
            var onePastLastAssetIndex = AssetTypes[assetTypeIndex].OnePastLastAssetIndex;

            var assetIndex = firstAssetIndex;
            while (assetIndex < onePastLastAssetIndex)
            {
                print("--------------");
                @(Header.Assets + hha_asset_size * assetIndex) hha_asset(assetType) Asset;

                var firstTagIndex = Asset.FirstTagIndex;
                var onePastLastTagIndex = Asset.OnePastLastTagIndex;

                @(Header.Tags + hha_tag_size * firstTagIndex) hha_tag AssetTags[onePastLastTagIndex - firstTagIndex];

                assetIndex = assetIndex + 1;
            }
        }

        assetTypeIndex = assetTypeIndex + 1;
    }
};

Conclusion

Hey cmon, we haven't checked the other HHA:s, how about audio and fonts?

You probably know enough about BEdit to make these yourself now! Keep in mind that BEdit is very young and will probably break faster than it works, but it's still being developed.

This concludes the HHA version 0 BEdit overview, it's probably not close to a file format you'd experience in the wild but it showcases the current capabilities of BEdit.

Edited by Jens on Reason: for loops supported in 0.0.3
Reply to Thread
← Back to index