This is the final week of GSoC! Need to finish everything ASAP. Instead of putting a report for this week, I will put up a complete final report which covers everything I did during these three months. Keep look out for that!
This is the final week of GSoC! Need to finish everything ASAP. Instead of putting a report for this week, I will put up a complete final report which covers everything I did during these three months. Keep look out for that!
I have been trying to fix the errors from the last week and complete the Single Output Mixer Pipeline. This cover week 11, the week from 29th July to 5th August.
Major tasks accomplished this week:
platform.request("hdmi_out", 1), because these pins aren’t used.
Next Week’s task:
Finish everything asap!
As I write this, I just noticed only two weeks (plus some more days) left in the official coding period of Google Summer of Code, some much time has already passed, hoping to complete the milestones by that time. This week’s focus was to first implement a static mixing of inputs, and once that is figured out in hardware, change the multiplier value from firmware to do dynamic mixing.
This work pertains to work done from 23rd July to 29th July.
Adding modules to Video Pipeline:
Last week I was adding my defined modules for float arithmetic the input video pipeline, that is in the hdmi_in files. Though this was a good test to check the working, the modules are supposed to be added to output pipeline. (gateware/hdmi_out/phy.py).
Major tasks done this week,
So apart from the usual payload signals, the source and sink of a pipelined module have four other signals defined, which control the flow of packet from source of master to sink of slave. These signals are namely stb, ack, sop, eop. Information about stb and ack can be found here. The other two, sop and eop, refer to start of packed and end of packet. So we basically want to know the equivalent connections for Record.connect()
# This is the Record.connect() method Record.connect(ycbcr2rgb.source, rgb2rgb16f.sink), def rgb_layout(dw): return [("r", dw), ("g", dw), ("b", dw)] # This is alternate way of doing equivalent connections rgb2rgb16f.sink.r.eq(ycbcr2rgb.source.r), rgb2rgb16f.sink.g.eq(ycbcr2rgb.source.g), rgb2rgb16f.sink.b.eq(ycbcr2rgb.source.b), rgb2rgb16f.sink.stb.eq(ycbcr2rgb.source.stb), ycbcr2rgb.source.ack.eq(rgb2rgb16f.sink.ack), rgb2rgb16f.sink.sop.eq(ycbcr2rgb.source.sop), rgb2rgb16f.sink.eop.eq(ycbcr2rgb.source.eop),
More information about significance and implementation of these signals is in this document (page 23).
Things seemed to be mixing well after this. Though there is some alignment problem due to timing inconsistency, which I need to figure out. For static testing I hardwired the mult with 0.5 value and connected output from HDMI_OUT0 and HDMI_OUT1 to floatadd unit of HDMI_OUT0. Later added a CSRStorage register, to dynamically vary the multiplier value from firmware to create a fade like effect. Added supporting functions to ci.c and other functions for maintaining timing.
Continuing with my regular weekly report, I am now in 9th week of Google Summer of Code. My regular college semester has started from this week and I am required to go to lab for my thesis work, so there haven’t been a lot of updates here. Hoping to complete a lot of things during the upcoming weekend. This report pertains to work done from 16th to 22nd July.
First half of the week was spent fixing errors and issues in previous work. One issue I was facing since a long time was the regarding testing and porting to litex. Like always the errors were some silly mistake on my part, and took longer than required to debug.
The idea was to use the pipelined hardware block to compute floating point multiplication of two numbers in hardware. The numbers were given through firmware by using CSR registers defined in hardware. After correctly initialising floatmult datapath in target/opsis_base.py, and adding the relevant csr functions in ci.c, I expected the output to be correct but somehow input was not transferring across. Which I thought was because incorrect implementation of CSR registers as wires, but it turned out way of connecting at input side was incorrect. # Incorrect way self.comb += [ self._float_in2.storage.eq(sink.in1), self._float_in2.storage.eq(sink.in2), self._float_out.status.eq(source.out) ] # Correct way self.comb += [ sink.in1.eq(self._float_in1.storage), sink.in2.eq(self._float_in2.storage), self._float_out.status.eq(source.out) ]
I was facing an error while porting my floating point arithmetic migen code to litex for review. Command line output of that error, link. Since this was coming up only while converting to litex, I thought there was some error with litex functionality. Later found out that this was because int variables being generated using numpy functions and hence the int variables were of numpy.uint type which was not correctly recognise by Signal object.
Although I had started working on run length encoding things after this, my mentor mithro suggested that I should first look at implementing mixer equation using a static mask for now.
The first step was to add all the submodules to video pipeline to check that they follow the timing constraints. I found that the existing color space conversion was done in gateware/hdmi_in/analysis.py in the FrameExtraction class. For testing purpose I added the rgbtofloat16 module and reverse module to this pipeline, connecting output of one to input to another in the hope of getting the exact same input. This also meant that the float16 color space conversion modules were tested in the hardware for the first time. The next step was to subsequently test floatmult and floatadd units by adding them in pipeline.
There were some issues in uploading the bit stream, which I eventually circumvented and here is a link to picture output when all the things are added in video pipeline. The picture is not too sharp, and I feel that there is a missing color in the gradient. I am gonna check this for static image in simulation.
Issue: The generated bitstream (which took close to 20 minutes each time), once uploaded didn’t work and the USB connection was even lost after this. My mentor mithro suggested that I should use the lightweight videomixer target, ($echo TARGET=video), and the problem might be because of FPGA resetting FX2 IC after it boots. This was basically bypassing the problem.
Right now I am trying to figure out where exactly to add video mixer equation in the bunch of gateware python files. I think I should add this at output side, because in that way I can operate on pattern input as well. The idea is that I will setup the standard video pipeline with one input for floatadd left open, for hdmi_out as well. And in the target/opsis_video.py file I will connect the unconnected input of floatadd of hdmi_out0 to output of floatmult of hdmi_out1 and vice versa. Once this is figured out, I will add CSR register to set the multiplier value from firmware.
This was a short week as I had to go out of town for three days. I have mostly worked on floating point conversions and arithmetic modules this week.
Float 16 Arithmetic: link
Float 16 Color Space Conversion: link
After a bit of break during week 6, as I was unwell, I am back to my standard GSoC schedule now. There were a lot of things left to be reviewed from last two weeks so most of the time this week went in getting code reviewed and making the required changes.
This work is continued from last week. I had defined a multiplier which worked for normal values, but gave incorrect results when the output is in the subnormal range of float16 variable. This was fixed by adding some extra conditions in the code. To test and debug the code efficiently added functions to convert unsigned int16 data type (the format in which data is streamed) to standard float and binary representation. This also helped to effectively verify the code for various test cases. This was code was tested to work fine on simulation, to test this on hardware, few more things were added to gateware and firmware to test the working.
In gateware, first added CSR storage and status registers. CSR is added as a part of bus support in migen and designed for accessing configuration and status registers of cores from software. Figured out a way to correctly initialize the FloatMult module in opsis_base, and checked the added functions in csr.h file for accessing operands and results of float multiplier. These functions for writing to multiplier operands and reading from multiplier outputs were added to ci.c file in lm32 firmware.
While testing this I figured out that, when output is held to a constant value in gateware code, it shows up correctly in firmware output. But when a previous stage’s output is held to constant value output remains as zero, which is the reset value. I suspect that in some part of initialization of CSR registers, something or other is inferred as an wire and not holding its value. I assume this is a trivial mistake in initializing something in migen.
I had also started writing code for adders likewise. Still a work in progress, hoping to get back to this once all the existing things get reviewed.
This pull request is in review since a lot of time now, hopefully in the final stages now. Apart from many other things I got to learn about usage of volatile data type in C. I had based my heartbeat code upon variables defined by pattern.c file, which used a volatile integer for framebuffer variable. Defining a variable volatile is required when we want the memory access to be sequential in the manner defined in code. But this was not the case here, not in pattern file as well, logged a github issue for this, which describe this issue in detail.
This week’s work on heartbeat involved cleaning up parts of code and adding documentation. After incorporating _florent_’s suggestions I no longer needed to make those changes hdmi_in and pattern c files. Thus, I won’t be needing to change those in this pull request which is a good practice to track changes. Also added documentation for setting desired heartbeat frequency. Link
Continuing with the review going on, made small changes here and there to improve the overall feel of the document and make it more easier for beginners in the field to understand. Added a whole bunch of links and added a github link to codes for generating the synthesis report results. This documentation currently explains in detail the FPGA mapping of a float16 multiplier, also adding a similar description for FPGA mapping of a float adder.
Ported this design to litex for merging with _florent_’s litevideo repository. Code review underway at this pull request. After _florent_’s suggestion I might add a look up table for this conversion instead of doing this using pipelined hardware and other things. I had earlier considered this but wasn’t sure about the available memory space and bandwidth. The size of look up table for uint8 to float16 will be a lookup table of size (16bits)*(256 locations) and (8bits)*(65536 locations), this can be further reduced if we cleverly consider the range of float16 number. The former is still okay, but latter seems a bit expensive.
After several weeks of reviews and comments on my work, my mentor, mithro added most of the code required for the run length encoding. I spent most of the time of a day trying to understand the code. Both the python and the C code used a lot of things here and there and I couldn’t imagine myself writing a code like that. I was supposed to work further on the code to add documentation and adding functions for image transformation. Still a work in progress. Will be adding the pull request soon.
Week 5 was mostly focussed on completing the midterm task. I had been so preoccupied with completing midterm tasks, coupled with a network outage during the crucial Thursday night during the midterm week, I was unable to present a weekly report last week. I continued with completing the midterm tasks till Monday this week. Feeling unwell since a few days since then. (maybe due to severe lack of sleep chasing midterm deadlines). Slight delay in adding this report as I was waiting to get it reviewed by my mentor.
Heartbeat Module: link
I had been dealing with glitchy output in the heartbeat module last week (Video). After incorporating suggestions from _florent_’s comments and some more independent bug fixing I was able to get a stable glitch free output on the screen. Here is a link to the pull request which contains marginal updates with each new commit.
One change I made was to modify the way framebuffer was allotted its base address, instead of copying it from input side (hdmi_in0_fi_base), I read the updated address from each of three designated outputs, (for eg hdmi_out0_fi_base). This still didn’t correct the glitch. But gave me the idea that my heartbeat glitch can be corrected by calling the hb_fill() at a much faster rate to match the HDMI input frame rate. I made some changes in the timing part of the code, with some other changes the heartbeat module works perfectly. The next step is to get it tested on someone else’s hardware and merge it with the main repository.
Float16 CSC conversion: code
Building up on my previous week’s work on float16 conversion, I was basically building up upon the existing pipelined structure designed for YCbCr color space conversion. While going through the code I realized how easy it was for me to understand and the elaborate test modules built to test the color space conversion using a lena.png image. My conversion modules for rgb2rgb16 and reverse were built up upon these.
The pipeline latency is two for both the conversions. I have basically described a single module for one conversion and reused it as submodule for complete rgb to rgb16 space conversion. The modules are designed specifically keeping in mind that input pixel is a 8 bit number and won’t necessarily work for a general integer. The input int8 numbers in the range 0 to 255 are mapped to 0-1 in their corresponding float conversion for added precision in that range in case of float. The design is implemented as described in the float16 design document.
For testing, there is already described test setup which uses streamer and logger to send and receive packed RGB data. To test the working of our hardware model, we need a standard model in described in software, this serves two purposes, one of comparing the hardware design with software model and other of converting the float data back to RGB pixel data to be later saved as an image.
Float16 Design document: link
Added FPGA mapping to design document which was missing until last time. This was added for the basic arithmetic operations and currently, covers multiplication. I have currently described a five stage pipeline for float16 multiplication and identified the components which are likely to use higher hardware resources. These are pretty simple components like 6 bit adders or 11 bit multipliers, I wrote simple Verilog codes for them, and took a look at the Xilinx XST synthesis report. I included both the hardware resource usage and the combinational delay of that element so that pipeline can be optimized for highest throughput.
Float16 Arithmetic: code
Since I had already worked with designing hardware in migen and completely described the pipelined hardware, this was pretty easy to describe. It is important to take care of edge cases here as they are described IEEE standard document. For example subnormal numbers or the description of NaN or Inf in float16 format.
The already existing test module based on images wouldn’t have been sufficient for the task here, as it will not necessarily cover all the edge cases. It is important to note the format in which bits are streamed for to the design under test hardware module. The streamer accepts integer data type, and corresponding 16 bit representation of that integer data type should correspond to the float16 value we are intending to send, and hence it became really difficult to debug things with this system. Hence I described two functions which convert to and from, float data type to int16 data type which is actually streamed and logged.
RLE Encoding and Decoding: code
After several code reviews, I finally have an object orientated implementation for generating an object of Repeat class. This class also defines generate() methods to return a list of repeated elements. Also defined is a Pixel class which is used to represent individual pixels, and returns a list of that pixel value, when the gen() function is called. Also added required documentation and python doc tests, to test the class methods and other functions.
Only one week left to complete midterm tasks. Heartbeat is integrated with firmware and ready for merge with upstream. Documentation on 16 bit float is ready, with a need for few minor edits.
Continuing with last week’s work on Opsis, I focussed on understanding how to upload a updated firmware without recompiling the gateware (which takes 20-30 minutes). I encountered some issues with correct FX2 USB IC modes. I will start will steps necessary to successfully load an updated firmware to opsis.
$ make load-gateware
$ ./<path>/hdmi2usb-mode-switch --mode=serial
Note that the make load-firmware command for uploading just firmware works only when FX2 USB IC is in serial mode. Toggle between jtag and serial mode once or twice to set the FX2 USB IC in correct serial mode.
$ make load-firmware
$ ./<path>/hdmi2usb-mode-switch --mode=jtag $ ./<path>/hdmi2usb-mode-switch --mode=serial
You should get updated pattern after this on the specified output. This means that firmware update was successful. You can also check the version information.
I have been working on adding the heartbeat functionality(#234) to the firmware for several weeks now. The C code was almost ready. Initially I was unable to load firmware with a new file for functions. This was because Makefile in firmware/lm32/ has predefined list of objects or the *.o files, which required including the heartbeat.o file manually.
Once I had figured out how how to load firmware, the first natural step was to understand how an individual pixel can be modified by making some changes in firmware. Tested this on pattern_fill_framebuffer() by making small changes to particular locations of framebuffer. Once I had figured out how to generate the required square at right bottom corner, I looked for ways to integrate it with timing functions to implement beating.
I enabled the pattern_service() function in processor_service() function to understand the timing related stuff. Once I had figured out the correct timing frequency, I added the timing code to my heartbeat function and tested it on pattern. This looked to be working fine. working video
Tested the same for hdmi_in0 as input, there were some glitches in the output of hdmi_in0, which maybe due to timing inconsistency or some kind of memory conflict. Added support for all the three possible inputs, hdmi_in0, hdmi_in1 and pattern. I added the three heartbeat functions in processor_service function. There is some glitch in output when all three run together. There might be better ways to integrate the heartbeat function. glichy video
Next, I made the required changes in ci.c files to get an option for enabling or disabling heartbeat at HDMI2USB> prompt.
The pixel data for RGB as well as YCbCr corresponds to 8 bit for each channel. Since we will be doing a number of operations on this pixel data it is important that data type gives us sufficient precision. Hence we convert the 8 bit pixel data to 16 bit float for further operations like gamma correction and linear mixing. I have documented the 16 bit float format in detail in the document link. This documents covers in detail the actual 16 bit float format, conversion and implementation of arithmetic operations.
I have already documented methods for this conversion in the design document linked in the previous section. I started with understanding the existing methods for conversion of rgb to ycbcr and other modules. This was first time I was dealing with a Migen code, so it was pretty alien to me in the first time.
The existing modules in csc are implementing the necessary conversion using pipelining, with some modules taking as much as 8 pipelined stages. The float conversion modules will follow very similar steps, and hence I started building up upon the existing modules by making changes wherever necessary. Although I had started with migen related things to learn something new and interesting, I went back to completing other things ongoing in my project.
Some of migen documentation I found online,
RLE Code: Couldn’t proceed due to lack of time this week. I will try to complete this first thing in the upcoming week.
So three weeks have already gone by and things seem to be picking up pace now. The midterm evaluation is two weeks from now and one of the thing was to have a clear set of goals for the mid term evaluation.
After having spent a lot of time in changing communication modes and toggling power supply I was finally able to get the Opsis working in its default mode. So last week I was stuck at a point, as I was unable to upload the bit stream. I was getting this error at make load-gateware.
$ make load-gateware .. .. ====== Building for: ====== Platform: opsis Target: opsis_hdmi2usb Subtarget: HDMI2USBSoC CPU type: lm32 =========================== Open On-Chip Debugger 0.10.0-dev-00248-gf3b38ff (2016-04-03-07:23) Licensed under GNU GPL v2 For bug reports, read http://openocd.org/doc/doxygen/bugs.html Warn : Adapter driver 'usb_blaster' did not declare which transports it allows; assuming legacy JTAG-only Info : only one transport option; autoselect 'jtag' Warn : incomplete ublast_vid_pid configuration jtagspi_program Info : usb blaster interface using libftdi Error: unable to open ftdi device: device not found
The error says that load-gateware script can’t find the opsis device. This was happening because the Opsis USB was not in the correct mode required for uploading bitstream. I used some of the scripts pointed out by CarlFK for changing the required mode. Following this, I was able to change the mode to “jtag” using this, but I got a different error at make load-gateware. (see the terminal output). When trying to read the DNA, using the same script, and got an incorrect DNA. This meant that the JTAG connection wasn’t set up correctly.
DNA = 110000001100000011000000110000001100000011000000110000001 (0x181818181818181)
After talking to my mentor mithro, I tried again with updated HDMI2USB-mode-switch repo. This also included a Makefile to install all the relevant tools and rules for correct JTAG connection. This is the HDMI2USB-mode-switch tool for doing mode-switch. I had a hard time completing the installation of required dependencies, due to having a <space> in path to directory. Lesson learnt: Never create folder names with spaces in them. The installation steps generated a Python file in HDMI2USB-mode-switch/conda/bin/, which was supposed to run with,
$ ./<path>/hdmi2usb-mode-switch --mode=jtag
After this I encountered another error. This happens when udev rules are not installed. These come along with the HDMI2USB-mode-switch installation makefile. I had missed this in the makefile while installing other tools. This was quickly resolved.
$ make load-gateware .. ====== Building for: ====== Platform: opsis Target: opsis_hdmi2usb Subtarget: HDMI2USBSoC CPU type: lm32 =========================== Open On-Chip Debugger 0.10.0-dev-00248-gf3b38ff (2016-04-03-07:23) Licensed under GNU GPL v2 For bug reports, read http://openocd.org/doc/doxygen/bugs.html Warn : Adapter driver 'usb_blaster' did not declare which transports it allows; assuming legacy JTAG-only Info : only one transport option; autoselect 'jtag' Warn : incomplete ublast_vid_pid configuration jtagspi_program Info : usb blaster interface using libftdi Error: unable to open ftdi device: inappropriate permissions on device!
One other common error I encountered was, this.
$ make load-gateware .. .. ====== Building for: ====== Platform: opsis Target: opsis_hdmi2usb Subtarget: HDMI2USBSoC CPU type: lm32 =========================== Open On-Chip Debugger 0.10.0-dev-00248-gf3b38ff (2016-04-03-07:23) Licensed under GNU GPL v2 For bug reports, read http://openocd.org/doc/doxygen/bugs.html Warn : Adapter driver 'usb_blaster' did not declare which transports it allows; assuming legacy JTAG-only Info : only one transport option; autoselect 'jtag' Warn : incomplete ublast_vid_pid configuration jtagspi_program Info : usb blaster interface using libftdi Info : This adapter doesn't support configurable speed Info : TAP xc6s.tap does not have IDCODE .. Info : TAP auto19.tap does not have IDCODE Warn : Unexpected idcode after end of chain: 21 0x00100000 Warn : Unexpected idcode after end of chain: 53 0x14049800 Warn : Unexpected idcode after end of chain: 85 0xfffffa20 Error: double-check your JTAG setup (interface, speed, ...) Error: Trying to use configured scan chain anyway... Error: xc6s.tap: IR capture error; saw 0x03 not 0x01 Warn : Bypassing JTAG setup events due to errors loaded file build/opsis_hdmi2usb-hdmi2usbsoc-opsis.bit to pld device 0 in 31s 968720us
The warning means that the JTAG firmware on the FX2 has gotten confused. Use the mode-switch tool to switch to serial mode and then back to the jtag mode like this.
hdmi2usb-mode-switch --mode=serial hdmi2usb-mode-switch --mode=jtag
This has to be repeated several times (up to three) to get the USB into correct jtag mode. So you basically have two terminal windows, one to change USB mode and other to do make load-gateware. One important observation was that if I followed certain steps than make load-gateware works without much repetitions.
Important step is to break the make load-gateware process when in serial mode by changing the mode back to jtag. Not sure why this is happening, but now won’t have to repeat a set of commands a lot of times.
After this make load-gateware succeeds, next connected to lm32 softcore using make connect-lm32, tried the bunch of options available to play around with. Added an image here, of the working setup. Still need to figure out how to load gateware and firmware independently.
A very simple way to understand run length encoding is to follow this wikipedia link. We are trying to do even better that this in the sense that we will have several levels of RLE applied over and over.
[AABBAABB]->[#2A#2B#2A#2B]->[repeat(2,A), repeat(2,B), repeat(2,A), repeat(2,B)]
Modified RLE for hdmi2usb-mask-gen
[AABBAABB]->[#2A#2B#2A#2B]->[repeat(2,A), repeat(2,B), repeat(2,A), repeat(2,B)] ->
[repeat( 2, ( repeat(2,A), repeat(2,B) ) ) ]
Last week I had explored the python PIL library to do basically convert an general RGB image to a list of 1s and 0s. This list was then used to generate the B/W image again. What remained was write a function to encode a list in a run length encoded form and decode it back from the encoded list to generate the image again.
As given in the Hardware Fader Design doc, we should have a repeat class, which can be then used encode multiple levels of run length encoding. In the case of multiple levels of RLE, decoding should be pretty easy, but encoding is not that straightforward.
I am sticking with single level RLE for now, the encoded values are stored in a 2D list, in which first column corresponds to pixel count and second column corresponds to pixel value. The rows are arranged sequentially, with mapping starting from left top to right bottom along horizontal lines.
Using this idea, I was able to do the complete python mask gen setup:
1. RGB–>B/W->Matrix(1s and 0s)->1D list
So the basic idea behind RLE is that the relevant masks when copied to and from memory will consume a lot of memory bandwidth which we are already lacking, hence the mask sequences are generated in an run length encoded form by the C firmware. These encoded values are then decoded by an equivalent hardware block, written in migen. All this should be under the timing constraints of a given frame rate.
Once some python functions were done, next step was to write the C firmware.
The idea is to have function like,
struct wipe* generate_wipe(enum wipe_style type, int position, int length)
This should do two things,
I have done the two things independently, there is still some issues with integrating them together in a function. The problem is mostly regarding passing a 2D array or a struct array in function.
Timer configuration in lm32: Timer is one of the many peripherals of the lm32 cpu core, as seen the block diagram figure. The C firmware can access the timer parameters using CSR, or specifically the csr.h file. CSR ( short for Control and Status Register) is a way for lm32 firmware to connect with the existing hardware. Using this we can use data values generated in hardware directly via the C firmware.
We find that several timer parameters are stored from location 0xe0002000 onwards. There are two timing functions defined in <time.c> (or its corresponding header <time.h>). One is time_init(), used in main.c to initialize the timer. Other is elapsed(), it counts if the timer has elapsed a certain number of ticks.
While exploring the implementation of timing function, it was noted that the timer functions counts down and not up. It basically is a counter which counts down at the trigger of system clock. The timer also has an interrupt which goes off when timer_value is 0.
Timer0 functions in csr.h
|Function Name||Function task||Used in|
|timer0_load||Timer value when not enabled, this is the initial timer value when just started||time_init()|
|timer0_reload||This is the value timer reloads to when it reaches zero.||time_init(), elapsed()|
|timer0_en||Enables the down counting of timer||time_init()|
|timer0_update_value||Updates the timer value from timer block to CSR registers||elapsed()|
|timer0_value||This gives the current value of timer||elapsed()|
Block diagram of lm32 and its peripherals. link
Opsis Board: My Opsis board was supposed to be shipped on 28th May, but the Numato guys were able to ship the board by 25th of May so I received my Opsis board this week itself. Yay!
I figured I didn’t have the relevant cable to initiate the testing, so I brought a bunch of cables (HDMI, USB B type, Adapter) for the Opsis connections. Later found out that power adapter is a bit shaky and sometimes disconnects due to nearby disturbances (Need to replace!). Also arranged for a HDMI monitor through a friend, in exchange for my VGA monitor. I was told that the current firmware might not work with VGA to HDMI converters so this had to be done.
The Opsis was preloaded with color bar pattern on both the HDMI outputs. This was easily tested. Next was to figure out USB connection mode. The first thing was to plug in the USB and figure out what ID the board appears on. Refer to this link for list of USB IDs. Among the several USB modes there are two USB modes relevant here,
|Mode||Vendor ID||Product ID||Device ID||Serial No|
|HDMI2USB.tv||0x2A19||0x5442||See Device ID table||Device MAC|
I was initially getting the Older ixo-usb-jtag USB mode, this was corresponding to the Van Ooijen Technische Informatica in the list of vendor names. The dmesg output for this mode looked like this,
$ dmesg | grep -i USB [68853.645703] usb 1-4: new high-speed USB device number 35 using xhci_hcd [68853.774157] usb 1-4: config 1 interface 0 altsetting 0 bulk endpoint 0x81 has invalid maxpacket 64 [68853.774164] usb 1-4: config 1 interface 0 altsetting 0 bulk endpoint 0x2 has invalid maxpacket 64 [68853.774518] usb 1-4: New USB device found, idVendor=16c0, idProduct=06ad [68853.774523] usb 1-4: New USB device strings: Mfr=1, Product=2, SerialNumber=3 [68853.774527] usb 1-4: Product: USB-JTAG-IF [68853.774530] usb 1-4: Manufacturer: ixo.de [68853.774532] usb 1-4: SerialNumber: hw_opsis
I figured this happens when USB is already connected during power cycle. So I disconnected the USB during the next power cycle, waited for about 20 seconds in the OFF mode and then turned ON, followed by connecting the USB. I found the HDMI2USB.tv mode using dmesg. Something like this.
$ dmesg | grep -i USB [68680.927026] usb 1-4: new high-speed USB device number 28 using xhci_hcd [68681.055627] usb 1-4: config 1 interface 2 altsetting 0 endpoint 0x81 has an invalid bInterval 64, changing to 10 [68681.056185] usb 1-4: New USB device found, idVendor=2a19, idProduct=5442 [68681.056191] usb 1-4: New USB device strings: Mfr=1, Product=2, SerialNumber=3 [68681.056195] usb 1-4: Product: HDMI2USB.tv - Numato Opsis Board [68681.056198] usb 1-4: Manufacturer: TimVideos.us [68681.056200] usb 1-4: SerialNumber: ffffd88039570494 [68681.056896] uvcvideo: Found UVC 1.00 device HDMI2USB.tv - Numato Opsis Board (2a19:5442) [68681.057778] cdc_acm 1-4:1.2: ttyACM0: USB ACM device
Note the Vendor ID and Product ID as mentioned in the table and found here.
This was the mode I was supposed to be in. I had already built the bit stream for Opsis from the latest github repo, the make load-gateware was showing some errors. Tried the solution suggested by CarlFK. Still the Opsis doesn’t seem to load the bit stream. This will be continued later in the week.
Python PIL Library: One of the other tasks scheduled for this week was to explore RLE encoding for 2D images using python. So the idea is that we can store the mask data in the memory in a run length encoded form. Initially we assume that each of the mask values are binary, ie they take only 1 or 0. I have used the Python PIL library for this. The initial task was to get a black and white (and not greyscale) image which can represent the mask. First the image is converted to a grayscale image, then it is converted to black and white using a suitable threshold. This black and white image is equivalent to a mask. Using PIL library functions, I generate a list of 1’s and 0’s, which correspond to the mask data derived from the images.
This list can now be run length encoded using a suitable function. The actual run length encoding part will be done after this. Now the opposite conversion should also be possible, hence we use the list of 1’s and 0’s to generate the image back. The current code is inverting the color, this was done to verify the working of code. The next task is to figure out optimum run length encoding for the list of numbers. <Github Link>
Plans for next week: