Daily Snippet:22nd June 2016

Posted on by shashankgangrade
  • In gateware/csc
    • Added rgb2rgb16f.py for 8bit pixel to 16 bit float conversion
    • Added rgb16f2rgb.py for reverse
    • Added relevant functions int the corresponding test files
    • Still need to add python (not migen) function for float to int8 conversion
    • Can be tested after above functions are done
  • In HDMI2USB-mixing-mask
    • Added changes in class definitions and updated functions
    • More discussions with mithro tomorrow
  • In heartbeat pull request
    • Incorporating suggestions from enjoy-digital
    • Will add updates by tomorrow

Weekly Report: Week4

Posted on by shashankgangrade

Only one week left to complete midterm tasks. Heartbeat is integrated with firmware and ready for merge with upstream. Documentation on 16 bit float is ready, with a need for few minor edits.

Updated just the Opsis firmware

Continuing with last week’s work on Opsis, I focussed on understanding how to upload a updated firmware without recompiling the gateware (which takes 20-30 minutes). I encountered some issues with correct FX2 USB IC modes. I will start will steps necessary to successfully load an updated firmware to opsis.

  1. Last week I had discussed how to upload gateware to opsis using make load-gateware. It is important that we have a .bit bitstream file for uploading the gateware. Next do make load-gateware, this is what we had achieved last week. Make sure that the FX2 USB IC is in correct jtag mode for this. This steps step needs to be done before we hack around with firmware code, this is because Opsis might be shipped with a previous version of gateware and current firmware might not be compatible. 
    $ make load-gateware
  2. To test the if the firmware is correctly updated or not, I added some lines to pattern.c, in the pattern_fill_framebuffer() function. After this use HDMI2USB-mode-switch discussed last week to switch to serial mode. 

    $ ./<path>/hdmi2usb-mode-switch --mode=serial

    Note that the make load-firmware command for uploading just firmware works only when FX2 USB IC is in serial mode. Toggle between jtag and serial mode once or twice to set the FX2 USB IC in correct serial mode.

  3. Next type make load-firmware, this will compile any updated C files and load the updated firmware. If the compilation and build steps are successful, it opens the serial terminal and we get HDMI2USB> prompt

    $ make load-firmware
  4. At the HDMI2USB prompt, type reboot. It should say something like
    Booting from serial…
    sL5DdSMmkekro
    [FLTERM] Received firmware download request from the device.

    HDMI2USB> reboot
  5. If you get “[FLTERM] Got unknown reply ‘T’ from the device, aborting“ then there has been an error downloading via the serial port. To resolve this, we need to switch to jtag and then back to serial mode of the FX2 USB IC using the HDMI2USB-mode-switch tool. This is similar to what we did for uploading gateware which used to jtag usb mode.

    $ ./<path>/hdmi2usb-mode-switch --mode=jtag
$ ./<path>/hdmi2usb-mode-switch --mode=serial

You should get updated pattern after this on the specified output. This means that firmware update was successful. You can also check the version information.

Adding heartbeat functionality to firmware

I have been working on adding the heartbeat functionality(#234) to the firmware for several weeks now. The C code was almost ready. Initially I was unable to load firmware with a new file for functions. This was because Makefile in firmware/lm32/ has predefined list of objects or the *.o files, which required including the heartbeat.o file manually.

Once I had figured out how how to load firmware, the first natural step was to understand how an individual pixel can be modified by making some changes in firmware. Tested this on pattern_fill_framebuffer() by making small changes to particular locations of framebuffer. Once I had figured out how to generate the required square at right bottom corner, I looked for ways to integrate it with timing functions to implement beating.

I enabled the pattern_service() function in processor_service() function to understand the timing related stuff. Once I had figured out the correct timing frequency, I added the timing code to my heartbeat function and tested it on pattern. This looked to be working fine. working video

Tested the same for hdmi_in0 as input, there were some glitches in the output of hdmi_in0, which maybe due to timing inconsistency or some kind of memory conflict. Added support for all the three possible inputs, hdmi_in0, hdmi_in1 and pattern. I added the three heartbeat functions in processor_service function. There is some glitch in output when all three run together. There might be better ways to integrate the heartbeat function. glichy video

Next, I made the required changes in ci.c files to get an option for enabling or disabling heartbeat at HDMI2USB> prompt.

Documentation of 16 bit float

The pixel data for RGB as well as YCbCr corresponds to 8 bit for each channel. Since we will be doing a number of operations on this pixel data it is important that data type gives us sufficient precision. Hence we convert the 8 bit pixel data to 16 bit float for further operations like gamma correction and linear mixing. I have documented the 16 bit float format in detail in the document link. This documents covers in detail the actual 16 bit float format, conversion and implementation of arithmetic operations.

Adding support for rgb16f to/from xxx in gateware/csc

I have already documented methods for this conversion in the design document linked in the previous section. I started with understanding the existing methods for conversion of rgb to ycbcr and other modules. This was first time I was dealing with a Migen code, so it was pretty alien to me in the first time.

The existing modules in csc are implementing the necessary conversion using pipelining, with some modules taking as much as 8 pipelined stages. The float conversion modules will follow very similar steps, and hence I started building up upon the existing modules by making changes wherever necessary. Although I had started with migen related things to learn something new and interesting, I went back to completing other things ongoing in my project.

Some of migen documentation I found online,

RLE Code: Couldn’t proceed due to lack of time this week. I will try to complete this first thing in the upcoming week.

Upcoming Week’s Task

  • Adding conversion modules for 16 bit float conversion in csc module.
  • Add migen code for multiplication
  • Cleanup of RLE codes
  • Update design documentation

Weekly Report: Week3

Posted on by shashankgangrade

So three weeks have already gone by and things seem to be picking up pace now. The midterm evaluation is two weeks from now and one of the thing was to have a clear set of goals for the mid term evaluation.

Opsis Board finally works!

After having spent a lot of time in changing communication modes and toggling power supply I was finally able to get the Opsis working in its default mode. So last week I was stuck at a point, as I was unable to upload the bit stream. I was getting this error at make load-gateware.

$ make load-gateware
..
..
====== Building for: ======
Platform:  opsis
Target:    opsis_hdmi2usb
Subtarget: HDMI2USBSoC
CPU type:  lm32
===========================
Open On-Chip Debugger 0.10.0-dev-00248-gf3b38ff (2016-04-03-07:23)
Licensed under GNU GPL v2
For bug reports, read
    http://openocd.org/doc/doxygen/bugs.html
Warn : Adapter driver 'usb_blaster' did not declare which transports it allows; assuming legacy JTAG-only
Info : only one transport option; autoselect 'jtag'
Warn : incomplete ublast_vid_pid configuration
jtagspi_program
Info : usb blaster interface using libftdi
Error: unable to open ftdi device: device not found

The error says that load-gateware script can’t find the opsis device. This was happening because the Opsis USB was not in the correct mode required for uploading bitstream. I used some of the scripts pointed out by CarlFK for changing the required mode. Following this, I was able to change the mode to “jtag” using this, but I got a different error at  make load-gateware. (see the terminal output). When trying to read the DNA, using the same script, and got an incorrect DNA. This meant that the JTAG connection wasn’t set up correctly.  

DNA = 110000001100000011000000110000001100000011000000110000001 (0x181818181818181)

After talking to my mentor mithro, I tried again with updated HDMI2USB-mode-switch repo. This also included a Makefile to install all the relevant tools and rules for correct JTAG connection. This is the HDMI2USB-mode-switch tool for doing mode-switch. I had a hard time completing the installation of required dependencies, due to having a <space> in path to directory. Lesson learnt: Never create folder names with spaces in them. The installation steps generated a Python file in HDMI2USB-mode-switch/conda/bin/, which was supposed to run with,

$ ./<path>/hdmi2usb-mode-switch --mode=jtag

After this I encountered another error. This happens when udev rules are not installed. These come along with the HDMI2USB-mode-switch installation makefile. I had missed this in the makefile while installing other tools. This was quickly resolved.

$ make load-gateware
..
====== Building for: ======
Platform:  opsis
Target:    opsis_hdmi2usb
Subtarget: HDMI2USBSoC
CPU type:  lm32
===========================
Open On-Chip Debugger 0.10.0-dev-00248-gf3b38ff (2016-04-03-07:23)
Licensed under GNU GPL v2
For bug reports, read
    http://openocd.org/doc/doxygen/bugs.html
Warn : Adapter driver 'usb_blaster' did not declare which transports it allows; assuming legacy JTAG-only
Info : only one transport option; autoselect 'jtag'
Warn : incomplete ublast_vid_pid configuration
jtagspi_program
Info : usb blaster interface using libftdi
Error: unable to open ftdi device: inappropriate permissions on device!

One other common error I encountered was, this. 

$ make load-gateware
..
..
====== Building for: ======
Platform:  opsis
Target:    opsis_hdmi2usb
Subtarget: HDMI2USBSoC
CPU type:  lm32
===========================
Open On-Chip Debugger 0.10.0-dev-00248-gf3b38ff (2016-04-03-07:23)
Licensed under GNU GPL v2
For bug reports, read
    http://openocd.org/doc/doxygen/bugs.html
Warn : Adapter driver 'usb_blaster' did not declare which transports it allows; assuming legacy JTAG-only
Info : only one transport option; autoselect 'jtag'
Warn : incomplete ublast_vid_pid configuration
jtagspi_program
Info : usb blaster interface using libftdi
Info : This adapter doesn't support configurable speed
Info : TAP xc6s.tap does not have IDCODE
..
Info : TAP auto19.tap does not have IDCODE
Warn : Unexpected idcode after end of chain: 21 0x00100000
Warn : Unexpected idcode after end of chain: 53 0x14049800
Warn : Unexpected idcode after end of chain: 85 0xfffffa20
Error: double-check your JTAG setup (interface, speed, ...)
Error: Trying to use configured scan chain anyway...
Error: xc6s.tap: IR capture error; saw 0x03 not 0x01
Warn : Bypassing JTAG setup events due to errors
loaded file build/opsis_hdmi2usb-hdmi2usbsoc-opsis.bit to pld device 0 in 31s 968720us

The warning means that the JTAG firmware on the FX2 has gotten confused. Use the mode-switch tool to switch to serial mode and then back to the jtag mode like this.

hdmi2usb-mode-switch --mode=serial
hdmi2usb-mode-switch --mode=jtag

This has to be repeated several times (up to three) to get the USB into correct jtag mode. So you basically have two terminal windows, one to change USB mode and other to do make load-gateware. One important observation was that if I followed certain steps than make load-gateware works without much repetitions.

  1. mode=jtag, make load-gateware, this gives the above warning.
  2. mode=serial, make load-gateware, while this is in progress (this won’t work ideally)
  3. mode=jtag, the make load-gateware will fail as usb mode was changed during process.
  4. Now make load-gateware, this should work.

Important step is to break the make load-gateware process when in serial mode by changing the mode back to jtag. Not sure why this is happening, but now won’t have to repeat a set of commands a lot of times.

After this make load-gateware succeeds, next connected to lm32 softcore using make connect-lm32, tried the bunch of options available to play around with. Added an image here, of the working setup. Still need to figure out how to load gateware and firmware independently.

Run Length Encoding (RLE)

Github Link

A very simple way to understand run length encoding is to follow this wikipedia link. We are trying to do even better that this in the sense that we will have several levels of RLE applied over and over.

For example:

Simple Example
[AABBAABB]->[#2A#2B#2A#2B]->[repeat(2,A), repeat(2,B), repeat(2,A), repeat(2,B)]

Modified RLE for hdmi2usb-mask-gen

[AABBAABB]->[#2A#2B#2A#2B]->[repeat(2,A), repeat(2,B), repeat(2,A), repeat(2,B)] ->  
[repeat( 2, ( repeat(2,A), repeat(2,B) ) ) ]

Last week I had explored the python PIL library to do basically convert an general RGB image to a list of 1s and 0s. This list was then used to  generate the B/W image again. What remained was write a function to encode a list in a run length encoded form and decode it back from the encoded list to generate the image again.

As given in the Hardware Fader Design doc, we should have a repeat class, which can be then used encode multiple levels of run length encoding. In the case of multiple levels of RLE, decoding should be pretty easy, but encoding is not that straightforward.

I am sticking with single level RLE for now, the encoded values are stored in a 2D list, in which first column corresponds to pixel count and second column corresponds to pixel value. The rows are arranged sequentially, with mapping starting from left top to right bottom along horizontal lines.

Using this idea, I was able to do the complete python mask gen setup:
 1. RGB–>B/W->Matrix(1s and 0s)->1D list

  1. 1D list –(RLE encoded)–> [2xn] list with specific format
  2. [2xn] list –(RLE Decoded)–>1D list
  3. 1D list –> B/W image

So the basic idea behind RLE is that the relevant masks when copied to and from memory will consume a lot of memory bandwidth which we are already lacking, hence the mask sequences are generated in an run length encoded form by the C firmware. These encoded values are then decoded by an equivalent hardware block, written in migen. All this should be under the timing constraints of a given frame rate.

Once some python functions were done, next step was to write the C firmware.

The idea is to have function like, 

struct wipe* generate_wipe(enum wipe_style type, int position, int length)

This should do two things,

  1. Generate a 2D array corresponding the wipe style and current position as compared to length.
  2. Encode the mapped 1D array according to the run length encoding (as implemented in Python) in a suitable struct.

I have done the two things independently, there is still some issues with integrating them together in a function. The problem is mostly regarding passing a 2D array or a struct array in function.

Upcoming Week’s Task

  • Test heartbeat on hardware
  • Complete the C code for generating RLE mask from templates, cleanup hdmi2usb-mask-gen
  • Documentation on 16 bit binary representation
  • Conversion of RGB+YUV to 16bit float and reverse in Migen