DataView()

By thevirtuoid on September 7, 2021 • ( Leave a comment )

Hi everyone, and welcome to another exciting edition of Boring JavaScript! Today, we tackle the DataView() object.

Don’t like to read? Then watch the video.

A Rose by Any Other Name

DataViews are perfect for when you wish to map binary data to a series of data values.

For example, suppose you were reading in a series of data from the following binary file, which represents the animals in a zoo:

01 06 46 6C 75 66 66 79 D9 F2 26 42 04 00 02 04
46 69 64 6F 75 94 62 40 09 00 03 07 57 69 6C 65 
20 45 2E C1 1C 77 41 03 00 00 00 00 00 00 00 00
00

Say what? Ok – what you see above is the hexidecimal (base 16) representation of the data in the binary file. Each byte (a single number above, from 00 to FF) represents a certain description of a data point. In this case, there is a ordered format to the data above (trust me). This format is as follows:

Byte 0 – The Type of Animal. If Type === 0, then that is the end of the list
Byte 1 – The length (in characters) of the Name of the Animal
Bytes 2 through 2+nameLength-1 – The Name of the animal
Bytes 2+nameLength through 6+nameLength-1 – The weight of the animal, as a IEEE 32-bit floating point number
Bytes 6+nameLength through 8+nameLength-1 – The age of the animal, as an Unsigned 16-bit integer number.

Given the description above, the bytes can be ordered like this:

01 06 46 6C 75 66 66 79 D9 F2 26 42 04 00     // the first animal
02 04 46 69 64 6F 75 94 62 40 09 00                // the second animal
03 07 57 69 6C 65 20 45 2E C1 1C 77 41 03 00 // the third animal
00 00 00 00 00 00 00 00                                      // no more animals!

To read the byte stream above, we will use the DataView to map data to each of bytes, allowing us to return those seemingly random bytes into understandable data. This is what that code will look like:

import fs from 'fs';
import Animal from "./Animal.js";
const fileContents = fs.readFileSync('./zoo.data');
console.log(`\nName\tType\tWeight\t\t\tAge`);
console.log(`----\t----\t------\t\t\t---`);
const dataView = new DataView(fileContents.buffer, fileContents.byteOffset, fileContents.length);
let offset = 0;

let animal = new Animal(dataView, offset);
while (animal.type !== "") {
	console.log(`${animal.name}\t${animal.type}\t${animal.weight}\t${animal.age}`);
	offset += animal.size;
	animal = new Animal(dataView, offset);
}

Line 3 grabs all the data from the raw binary file, then Line 6 is our ‘DataView’ line. It takes three arguments (the last two being optional):

The buffer. This will be an ArrayBuffer, which is the storage for the actual data
Byte Offset. This will be where in the ArrayBuffer to start the mapping process. If not specified, the start of the ArrayBuffer (0) is used.
Size of Buffer. This will be the number of bytes to include in the DataView. If not specified, the entire contents of the ArrayBuffer will be used.

I didn’t have to specify ‘fileContents.byteOffset’ and ‘fileContents.length’ above – but I did anyway just for completeness sake.

Mapping The Course

The above creates the DataView, but how do you map JavaScript variables to it? Glad you asked! Below is a class that has been defined to extract the animal from the zoo. That is where the real magic happens. Let’s take a look:

export default class Animal {
	static TYPES = ['', 'Cat', 'Dog', 'Coyote'];

	type = '';
	name = '';
	weight = 0;
	age = 0;

	constructor(view, offset) {
		const type = view.getUint8(offset);
		this.type = Animal.TYPES[type];
		const nameLength = view.getUint8(offset + 1);
		this.name = getName(offset + 2, nameLength);
		this.weight = view.getFloat32(offset + 2 + nameLength, true);
		this.age = view.getInt16(offset + 6 + nameLength, true);

		function getName(offset, length) {
			if (length === 0) {
				return "";
			} else {
				offset += view.byteOffset;
				return String.fromCharCode.apply(null, new Uint8Array(view.buffer.slice(offset, offset + length)));
			}
		}
	}

	get size () {
		return 1 + 1 + this.name.length + 4 + 2;	// Bytes (type + sizeOfName + nameLength + weight + age)
	}
}

A little lengthy, but very straight forward. What we’re going to concentrate on is the constructor() of the class, which contains the logic of mapping the data values to the DataView. The DataView object contains helper methods for extracting all sorts of integers and floating point numbers from the data stream. We’re not going to cover them all here – only the ones that we need. But please do check out the MDN documentation on DataView to see what’s available to you.

In each of those helper methods, you will specify the starting offset within the DataView to get the data, and sometimes a flag indicating if you wish Little Endian or Big Endian notation (more on that in a second)

In our example, we have four data values.

Type (Line 10): This is a single byte value that is an Unsigned 8-bit Integer. Here, we use DataView.getUInt8() to get that value.
Size of Name (Line 12): This is a single byte value that is an Unsigned 8-bit Integer. Again, we use DataView.getUInt8() to the get that value.
The Name (Line 13): This is a string, and since there is no ‘string’ defined within the DataView spec, I had to write a function to extract the name (the ‘getName’ function).
Weight (Line 14): This is an 32-bit IEEE floating point number, and so we use DataView.getFloat32() to grab the number. Notice the second argument, ‘true’. This tells us to read in the data in ‘Little Endian’ format (more on that in a second).
Age (Line 15): This is an Unsigned 16 bit Integer, so we use DataView.getUint16(). Again, this has the second argument of ‘true’.

Using the above, we can now map out a certain order of bytes to match the requirements of our records.

What about ‘size’ (Line 27)? Since each record is a variable length, we need a way to tell the consuming app exactly how long this particular record is – in terms of the number of bytes in the DataView. So we set up a ‘getter’ function to do that. This way, the consuming app can use ‘size’ to get to the next record in the DataView.

Little Endian and Big Endian

Now it’s time to look at that second argument. It tells JavaScript to read in the data as ‘Little Endian’ or ‘Big Endian’ format. This is an old computer science term that tells you how to read in multi-byte numbers.

For example, a two-byte unsigned integer number is computed by multiplying one byte by 256 and adding in the second byte. The number 258 is stored in two bytes – ’01’ and ’02’, because 1 * 256 + 2 = 258.

When that number is stored, though, do you store the ‘1’ first and then the ‘2’, or the ‘2’ first? That’s what ‘Big Endian’ and ‘Little Endian’ is all about.

‘Big Endian’ stores the most significant byte first, then proceeds in order to the least significant byte. In our case the byte that is multiplied by 256 is stored first in memory.

‘LIttle Endian’ stores the least significant byte first, then proceeds in order to the most significant bytes. In our case, the byte that is multiplied by 1 is stored first in memory.

In our particular case, the data file is using Little Endian to specify multi-byte numbers. Since the default of the Endian argument (the second argument) is false, that means by default all the multi-byte numbers will be read in Big Endian format. Therefore, we need to set it to True in order to get what we want.