Bitwise Operations for the Average Developer

As a web developer, I haven’t thought much about binary and bitwise operations since early school days. I just didn’t see a place where shifting or XOR’ing bits would fit in my CRUD apps. That was until I stumbled upon a trick that would save me a lot of headache when dealing with database models.

The following code will be in JavaScript and SQL, but the same concepts will hold in almost any language.

What is binary?

First of all let’s see how binary data is written. If you already know how binary works, you can skip to the next section.

Binary Format

You probably know that binary is represented in zeros (0) and ones (1). Lets say we have a number 47 ,it can represented it as: $$2^5+2^3+2^2+2^1+2^0 = 47$$ To write it in binary we count exponents from right to left:

Put 1 if we have a 2^exponent
Put 0 if we don’t use that exponent

47 can be represented as 101111:

 1  0  1  1  1  1
 ↑  ↑  ↑  ↑  ↑  ↑
 2⁵ 2⁴ 2³ 2² 2¹ 2⁰

When coding, JavaScript needs a hint to know that the variable you’re defining is binary. To do that we add 0b at the start of the value like this:

let binary = 0b101111;

Which would represents 47:

let binary = 0b101111;
let decimal = 47;

binary === decimal; // true

Binary operations

Bitwise AND ( & ), OR ( | ), XOR ( ^ )

When applying a binary operations to two values, we apply the operator to each bit individually:

More details about binary operators can be found on MDN .

// AND (&)
const a = 5; // 0101
const b = 3; // 0011
a & b; // 0001

// OR (|)
const a = 5; // 0101
const b = 3; // 0011
a | b; // 0111

// XOR (^)
const a = 5; // 0101
const b = 3; // 0011
a ^ b; // 0110

Now that you know how to represent and manipulate binary data, we can get to our use-case!

Real world use-case for binary operations

Let’s image we have a database with a table users with a simplified user model, where can_buy property is a boolean that indicates if the user is able to buy or not on our app.

 id |        email        | can_buy
----+---------------------+----------
  1 | alice@example.com   | true
  2 | bob@example.com     | false
  3 | eve@example.com     | false
  4 | steve@example.com   | true

Now let’s imagine we want to add the ability to sell for some users. Easy right? Just add a column can_sell.

ALTER TABLE users ADD `can_sell` BOOLEAN DEFAULT FALSE;

This doesn’t seem as a big deal until you look at the consequences:

Setting a default for every row. The schema will be changed, and every row will be modified while the default data is written. Locking will occur until the operation is completed. For large tables this can become problematic.

Some time later, we’re asked to add another few columns can_bid, can_comment, can_respond, etc …

See how adding columns can become very tedious?

Binary operations to the rescue

What if we have a table with a numeric column that we can call permissions:

 id |       email        | permissions
----+--------------------+-------------
  1 | alice@example.com  |           0

Then we can model our permissions in a binary format. Each bit is assigned to a given boolean property, when it is set to 1 then that value is TRUE otherwise it is FALSE. For example:

00000001; // (1) can_buy
00000010; // (2) can_sell
00000100; // (4) can_comment
00001000; // (8) can_respond
//etc ...

So if we have user with permissions = 10 We can map it to:

00000010; // can_sell
// OR
00001000; // can_respond
00001010; // (10)

So our example user with permission 0b00001010

can_buy FALSE
can_sell TRUE
can_comment FALSE
can_respond TRUE

Now any time we need to add another boolean permission, we can map it to an unused bit. We don’t need to alter the table schema to hold more information.

Application code

With what we learned above we can write an application code that encodes/decodes the stored permissions number to an object with boolean properties.

const PERMISSIONS = {
  IS_BANNED: 0b1,
  CAN_LOGIN: 0b10,
  CAN_BUY: 0b100,
  CAN_SELL: 0b1000,
  IS_ADMIN: 0b10000,
};

const encodePermissions = ({
  isBanned,
  canLogin,
  canBuy,
  canSell,
  isAdmin,
}) => {
  let encoded = 0;

  encoded |= isBanned && PERMISSIONS.IS_BANNED;
  encoded |= canLogin && PERMISSIONS.CAN_LOGIN;
  encoded |= canBuy && PERMISSIONS.CAN_BUY;
  encoded |= canSell && PERMISSIONS.CAN_SELL;
  encoded |= isAdmin && PERMISSIONS.IS_ADMIN;

  return encoded;
};

const decodePermissions = encoded => ({
  isBanned: !!(encoded & PERMISSIONS.IS_BANNED),
  canLogin: !!(encoded & PERMISSIONS.CAN_LOGIN),
  canBuy: !!(encoded & PERMISSIONS.CAN_BUY),
  canSell: !!(encoded & PERMISSIONS.CAN_SELL),
  isAdmin: !!(encoded & PERMISSIONS.IS_ADMIN),
});

// Encode permissions as a number
const permissionsEncoded = encodePermissions({
  isBanned: false,
  canLogin: true,
  canBuy: false,
  canSell: false,
  isAdmin: true,
});

// Decode permissions to object
const permissionsObject = decodePermissions(permissionsEncoded);

/*
 Numeric: 18
 Binary:  10010
 Decoded: {
  "isBanned": false,
  "canLogin": true,
  "canBuy": false,
  "canSell": false,
  "isAdmin": true
}
*/

Database queries

We can also query the database and select using bitwise operators.

 id |       email        | permissions
----+--------------------+-------------
  1 | alice@example.com  |           0
  2 | bob@example.com    |           5
  3 | eve@example.com    |          10
  4 | steve@example.com  |          12

Let’s select all users who can comment:

SELECT * from users where (permissions::bit(8) & '00000100') = '00000100';
 id |       email        | permissions
----+--------------------+-------------
  2 | bob@example.com    |           5
  4 | steve@example.com  |          12

With that same logic, we can also select users who can't sell:

SELECT * from users where (permissions::bit(8) & '00000010') = '00000000';
 id |       email        | permissions
----+--------------------+-------------
  1 | alice@example.com  |           0
  2 | bob@example.com    |           5
  4 | steve@example.com  |          12

Conclusion

I hope you enjoyed the read, and learned something new; Just because we’re not explicitly using binary data, doesn’t mean you can’t take advantage of binary operators.

This particular use-case can be useful to avoid locking large tables when adding new columns. The inconvenience is that we need to maintain a dictionary of what each bit maps to. Which could be error prone.

If you have other use-cases for the average developer, you can share them with a comment!