Wednesday, May 27, 2015

Using the BPG image format on Android

There's a new image format in town and it's called BPG (Better Portable Graphics). Well, it's not so new, as it's been introduced in 2014, but it's new enough that you can find little information about integration with different platforms.
For academic/experimental purposes I've made a small Android application to see the format in practice.

A link to a working example will be provided at the end of the article.

The first step would be to get the source code of the libbpg library, which can be downloaded from here. It comes in the form of a tar.gz. It does not conform to the usual configure-make-make install pattern, it just contains the sources and some makefiles. You have to figure out the dependencies by yourself.
Speaking of dependencies, only the test applications have dependencies on libpng, SDL, and other stuff, the libbpg core library is plain old C and has no dependencies.

To get started on Android, one first has to compile libbpg. There are a lot of tutorials on how to build C code for Android so I won't go into details about this. I for one used eclipse, following the steps here.

In order to display an image in Android, I've used an ImageView component into which I loaded the content of a Bitmap, which had been filled with the decoded data from libbpg.
Now, the sources of libbpg contain some example applications for decoding and encoding images from file (bpgenc.c and bpgdec.c). Since I want to use decoding from a buffer and the examples were only decoding to PPM and PNG, I had to write my own decoding function which transformed the data to a BMP format, in order to be consumed by the ImageView component.

Decoding to BMP looks like this:
#pragma pack(1)  // ensure structure is packed
typedef struct
{
    uint16_t  bfType;
    uint32_t bfSize;
    uint16_t  bfReserved1;
    uint16_t  bfReserved2;
    uint32_t bfOffBits;
} BITMAPFILEHEADER;

typedef struct {
  uint32_t biSize;
  int32_t  biWidth;
  int32_t  biHeight;
  uint16_t biPlanes;
  uint16_t biBitCount;
  uint32_t biCompression;
  uint32_t biSizeImage;
  int32_t  biXPelsPerMeter;
  int32_t  biYPelsPerMeter;
  uint32_t biClrUsed;
  uint32_t biClrImportant;
} BITMAPINFOHEADER;
#pragma pack(0)  // restore normal structure packing rules

static void bmp_save_to_buffer(BPGDecoderContext *img, uint8_t** outBuf, unsigned int *outBufLen)
{
    BPGImageInfo img_info_s, *img_info = &img_info_s;
    int w, h, y, size_of_line, bufferIncrement, x;
    uint8_t *rgb_line;
    uint8_t swap;
    BITMAPFILEHEADER header;
    BITMAPINFOHEADER info;

    memset(&header, 0, sizeof(BITMAPFILEHEADER));
    memset(&info, 0, sizeof(BITMAPINFOHEADER));

    bpg_decoder_get_info(img, img_info);

    w = img_info->width;
    h = img_info->height;
    // find the number of padding bytes
    int padding = 0;
    int scanlinebytes = w * 3;
    while ( ( scanlinebytes + padding ) % sizeof(uint32_t) != 0 ){
        padding++;
    }
    // get the padded scanline width
    size_of_line = scanlinebytes + padding;
    rgb_line = malloc(size_of_line);
    if(NULL == rgb_line){
        printf("FAILED to allocate \n");
        return;
    }

    //prepare the bmp header
    header.bfType = 19778;
    header.bfSize = sizeof(BITMAPFILEHEADER)+sizeof(BITMAPINFOHEADER);
    header.bfOffBits = sizeof(BITMAPFILEHEADER)+sizeof(BITMAPINFOHEADER);
    //prepare the bmp dib header
    info.biSize = sizeof(BITMAPINFOHEADER);
    info.biWidth = w;
    info.biHeight = h;
    info.biPlanes = 1;
    info.biBitCount = 24;
    info.biSizeImage = w*h*(24/8);

    *outBufLen = size_of_line * h + sizeof(header) + sizeof(info);
    *outBuf = malloc( *outBufLen );
    if(NULL == *outBuf){
        printf("FAILED to allocate \n");
        free(rgb_line);
        return;
    }
    memset(*outBuf, 0, *outBufLen);
    //copy the header and info first
    memcpy(*outBuf, &header, sizeof(header));
    memcpy(*outBuf+sizeof(header), &info, sizeof(info));

    bpg_decoder_start(img, BPG_OUTPUT_FORMAT_RGB24);
    bufferIncrement = size_of_line;
    for (y = 0; y < h; y++) {
        bpg_decoder_get_line(img, rgb_line);

        // RGB needs to be BGR
        for (x=0; x < size_of_line; x+=3){
            swap = rgb_line[x+2];
            rgb_line[x+2] = rgb_line[x]; // swap r and b
            rgb_line[x] = swap; // swap b and r
        }
        memcpy( (*outBuf)+*outBufLen-bufferIncrement, rgb_line, size_of_line);

        bufferIncrement += size_of_line;
    }
    free(rgb_line);
}
As an explanation for the above code: we get the decoded data in PPM format, we need to transform it to BMP. The BMP is just an upside down PPM file, with some headers, so, we need to add the headers, invert the data and flip the RGB bytes. As a guide, I used this great tutorial on working with BMP data.
The hardest part being done, we need to add some JNI glue to the application. The JNI interface methods are:
public class DecoderWrapper {
 
 public static native int fetchDecodedBufferSize(byte[] encBuffer, int encBufferSize);
 
 public static native byte[] decodeBuffer(byte[] encBuffer, int encBufferSize);

}
The fetchDecodedBufferSize is not used yet, so the only method used is decodeBuffer which will return the decoded BMP in a byte buffer.
The actual JNI code is pretty simple, so I will not dump it in the article. It can be examined in the project link.

An example of using the function in Java:
public Bitmap getDecodedBitmap(int resourceId){
    Bitmap bm = null;
    InputStream is = getResources().openRawResource(resourceId);
    try{
     byte[] byteArray = toByteArray(is);
     byte[] decBuffer = null;
     int decBufferSize = 0;
     decBuffer = DecoderWrapper.decodeBuffer(byteArray, byteArray.length);
     decBufferSize = decBuffer.length;
     if(decBuffer != null){
      bm = BitmapFactory.decodeByteArray(decBuffer, 0, decBufferSize);
     }
    }
    catch(IOException ex){
 Log.i("MainActivity", "Failed to convert image to byte array");
    }
    return bm;
}
I'm using an embedded BPG resource for my tests, but in practice, some data received from a server would more practical.

The final application looks like this:
There is some information related to loading times (decompression and rendering) and image size at the bottom.

The full project can be found here: https://github.com/alexandruc/android-bpg.

As an improvement, data transfer from C to Java might be optimized by using jnigraphics so that the buffers are not duplicated. I will maybe add this to a future version.


14 comments:

  1. Excellent work ! bravo
    It worked for me successfully without any bug !
    What about compression BPG on Android ? is it laborious to do ?

    ReplyDelete
    Replies
    1. Glad it could help you.
      For encoding you could check out the example on the official bpg page. They have some utilities for encoding.
      I don't recommend encoding on mobile as it is a cpu intensive operation.

      Delete
    2. It really will help me alot. Big Thanks !
      I'm agree for the CPU consummation in the case of BPG encoding. However, nowadays the phones capacity has increased ! to know if it will be enough! that's what I'll try to check soon with the BPG encoding :)

      Delete
  2. I tried to add new images​ ​for decoding.
    It works for some images but not for others. It doesn't work​ exactly for images with odd size ​! In this case, the BitmapFactory.decodeByteArray() returns NULL. So, my bitmap image is null too.
    Have​ you any idea please ?​
    Thanks for your help​​.

    ReplyDelete
    Replies
    1. Hi Yasmine,
      It was a bug in the bmp conversion. I've fixed it. Please pull the latest sources.

      Delete
    2. Hi Alexandru,
      It did not work yet, but I fixed the bug ^^
      You forgot to compute the number of padding bytes when you convert the bmp image. You should just replace this line in your code :
      ******************
      size_of_line = w%2 ? (3*w+1) : 3*w;//account for images with odd resolution)
      ******************
      by :
      ******************
      // find the number of padding bytes
      int padding = 0;
      int scanlinebytes = w * 3;
      while ( ( scanlinebytes + padding ) % 4 != 0 ) // DWORD = 4 bytes
      padding++;
      // get the padded scanline width
      size_of_line = scanlinebytes + padding;

      ******************
      And it will work for any image size :)

      Delete
    3. If you want to check, here you can test a kind of image which posed a problem for me :
      http://www.gstatic.com/webp/gallery/2.jpg
      size : 550*404

      Delete
    4. Your fix seems to cover all cases. I have tested and merged it. Thanks.

      Delete
  3. Hi Alexandru,
    I try to use your library, and get this error on Asus based device
    http://pastebin.com/tVz0iTzt

    while it's ok when i run on emulator.

    can you give me some advice on what i did is wrong or how to fix it?

    Thanks in Advance

    ReplyDelete
  4. Hi Alevandru,
    Nice works.
    But in few months Google has made it own IDE, called Google Studio.
    Can I use Google Studio bundle to open the project and compile it?
    Could you provide the sample apk?
    Thank you.

    ReplyDelete
  5. Thank you for this article!
    I tried to find a solution for React Native and succeed only because you wrote this.
    Also I made a module based on your solution, published it to github and npmjs, and of course mentioned you :)
    Thanks!

    https://www.npmjs.com/package/react-native-bpg
    https://github.com/nosshar/react-native-bpg
    https://github.com/nosshar/react-native-bpg-example

    ReplyDelete
    Replies
    1. Hi,
      I'm glad it helped you and thanks for the references.

      Delete