hpkbra
Last Updated: February 25, 2016
·
5.806K
· neutralino1
981494 10151466984546915 1613396205 o

Using bitmasks to store settings

We often need to store several booleans to determine a user's settings. Bitmasks are a good economic way to do so. The idea is to use a single integer where each bit represents one such boolean. This is better than saving each individual settings in a different database column.

Choose bit positions for each settings. Lets say we want to store notification settings.

NEWSLETTER = 0
COMMENTS = 1
FOLLOWER = 2

Now let's turn off all settings:

user.settings = 0

Now let's turn on comment notifications:

user.settings |= (1 << COMMENTS)

Let's see how it looks like in binary:

user.settings.to_s(2)
>>> "10"

The bit at position 1 (COMMENTS) was set to 1 while the rest hasn't changed.

The operation 1 << N places (shifts) the value 1 to the position N so that e.g.:

 (1 << 4).to_s(2)
>>> "10000"

Now | and & are the usual OR and AND operators but applied at a bit level so that :

(1 << 4 | 1 << 2).to_s(2)
>>> "10100"

We see that a |= 1 << N turns ON the N-positioned bit in the integer a.

The operator ~ inverts all the bits in a variable so that:

(~0b1010).to_s(2)
>>> "-110"

Which is all bits set to 1 except bit 1 and bit 3 if we talk about signed integers.

To turn a bit OFF, simply do

user.settings &= ~(1 << COMMENTS)

since ~(1 << COMMENTS) will have all bits ON except the one at position COMMENTS.

To check if a specific settings is ON, simply check

user.settings & (1 << COMMENTS) > 0
Say Thanks
Respond

15 Responses
Add your response

12471
01e524cc2a4c9ee03a4cf982d6683adf

This is great. A potential downfall is that you kind of lose the ability to query and report on those settings. "Show me users that have disabled followers" becomes a bit (ha) more difficult.

over 1 year ago ·
12472
981494 10151466984546915 1613396205 o

That's a fair point.
If I'm not mistaken, all flavours of SQL support bitwise operations don't they? Hence one could query with SELECT * FROM users WHERE settings & (1 << 3) > 0.
Isn't that right?

over 1 year ago ·
12498
187f3356c852d3962517834d20aa18f7 normal

@neutralino1, that is indeed THE solution for this. I love bitmasks too, they certainly make some tasks a lot easier (and save a lot of extra columns in your database).

BUT, you always got to be careful when using them, not just throw everything in there :P

over 1 year ago ·
12502
981494 10151466984546915 1613396205 o

Indeed, using bitmasks requires some serious testing.
And you sure can't use it for everything. In my case, I use it for a stupid collection of checkboxes.

over 1 year ago ·
12506
0d38ee92831618c24839dc002859c32f

This would have been the only way dev's stored things years back, when memory was an issue. Wouldn't it be nice to see how fast modern code could be if time was spent to speed it up.

over 1 year ago ·
12508
981494 10151466984546915 1613396205 o

@encodes Indeed.
I actually learned this technique while coding for the ATLAS experiment at CERN which contains thousands of electronic chips where both physical and logical space are limited resources.

over 1 year ago ·
12512
0d38ee92831618c24839dc002859c32f

@neutralino1 Yea, its definitely an underused technique. I used a similar technique on a micro-controller for a burglar alarm. With 8 zones, we need just 3 bytes to control whether they are enabled, alarmed and triggered. Then its just a matter of setting up alarms/switches and LED's to trigger these points.
Bitwise operators are just not taught in main stream anymore.

@mikeymike I think you will appreciate this one.

over 1 year ago ·
12526
Df794bc411b5637c13d3f2d5a8b1cffa

@encodes that's pretty neat!

over 1 year ago ·
12571
Photo on 08.01.2013 at 04.15

bitmaps in ruby? you gotta be kidding

over 1 year ago ·
12573
981494 10151466984546915 1613396205 o
over 1 year ago ·
12574
Photo on 08.01.2013 at 04.15
  1. If you're using bitmaps in-memory you need to allocate more memory for objects to work with them than the save you space.
  2. If you're using bitmaps in database you cannot work with this data in any sensible way (no indexes, no clean queries), so you use bitfield just as crippled storage format. Moreover each database has it's own, performant and indexable structures to store such data.

Not mentioning I'd never scarify system and code readability to save few bytes of space per record. Sounds like premature optimisation, eh?

In Ruby you get neither of performance, code readability, lower storage/memory space, data normalisation, good query interface. There is literally no advantage.

over 1 year ago ·
12575
981494 10151466984546915 1613396205 o

@sheerun I have a table dedicated to users settings with dozens of booleans columns and as many rows as users. That freaks me out. I much rather like a single integer field on the users table.
I never need to query on them, I only need to check a given record's settings.

Now to be clear, I never said people should use this technique blindly. Everyone should be sensible to how it fits there requirements. Just like everything else.
Also, I used ruby here to illustrate the point but the post was mostly intended to showcase the technique rather than the language used to apply it.

over 1 year ago ·
12643
7a1054e17f3c41c66b1b28d4d6b09379

I once worked at company which was licensing a product connected to a 3rd party proprietary database which we had almost zero control over. There was a table in particular that needed two additional integer type columns. With only one column "spare" in the table, I suggested we use a bit-weighted value to solve the problem.

It's not an old idea but one to remember "just-in-case".

I should also point out that code which dealt with these values was harder to read. Bit Shifting is an easy concept to understand but unless you're shifting bits all day, it's bound to slow down some developers and make them pause longer than they need to while fixing an issue in production.

Some convenience wrapper type methods would add value.

over 1 year ago ·
12655
2d59182a27674ee933d9fe0303454a65

Does endianness become a problem with this kind of approach?

over 1 year ago ·
12790
05ae1fd20ada15bfe629db803e4c00b7

When you have 65 roles+ ( 64 bits / 8 = 8 byte ), it will be wrong way. Because MySQL only have 8bytes ( BIGINT data type). How to fix it ?

over 1 year ago ·