Gennum driver

Hi,

I am FPGA designer at MAX IV and I would like to make some questions regarding to Gennum GN4124 solution. We are working on the ALBA Electrometer which uses a SPEC board version 4, but we have some issues regarding to PCIe express throughput.

The Alba Electrometer uses alin.py script to have access to internal FPGA registers. It seems that alin.py script uses spec_lib.py script and libspec.so driver: https://www.ohwr.org/project/fmc-adc-400k18b4cha-iso-sw/tree/master/alin

The issue that I have found out is that the FPGA has received a big amount of data when I try to write or read data from some internal FPGA register.

The figures below show a alin.py data write command (Figure 1) and a ChipScope debug (Figure 2) at p2l_des block (P2L deserializer) from gn4124_core block.
The Figure 2 shows a big amount of data coming directly from the Gennum GN4124, when I asked to write 0x000000AB data to the address 0x400A (0x1005), for instance.

(Figure 1)
image

(Figure 2)
figure_2

Maybe there is some issue at alin.py/spc_lib.py/libspec.so.

My questions are:

  1. We would like to make sure what driver do you recommend?

  2. Do you already have a solution that provides access to CSR and DMA interface? I’ve found a project example that uses rr.py/rawrabbit.c/rrlib.so files. Do you recommend this solution to have access to FPGA via PCI express? https://www.ohwr.org/project/gn4124-core/wikis/Documents/Project-Attachments

Regards,

Orlando

About the specific problem I feel that the issue is more lower level: FPGA or the GN4124 itself.
As you can see in spec-sw/blob/master/tools/speclib.c, there is no logic implemented in the speclib regarding read/write.

And after a super-quick look at alin, it does not look like there is anything there too.

We would like to make sure what driver do you recommend?

Today, the spec-sw is the only supported driver for the SPEC card.
Having said this, if you only access the card from user-space, the driver does not do much, everything is done by the Linux kernel. The spec driver is used to program the FPGA and configure some GPIO for interrupts and nothing more; in-kernel users will get other features but I understand that this is not your case.

Do you already have a solution that provides access to CSR and DMA interface?

Directly from userspace we don’t have any interface for DMA

I’ve found a project example that uses rr.py/rawrabbit.c/rrlib.so files. Do you recommend this solution to have access to FPGA via PCI express? https://www.ohwr.org/project/gn4124-core/wikis/Documents/Project-Attachments

Anything based on rawrabbit is not supported, it is an old project used for testing.

1 Like

Hi if I may add my two cents here;
A few years ago I too spent quite amount of time trying to make the DMA working on the spec board, only to no avail. The example code found on HDLmake web site only has the register read/write, no DMA. GN4124 chip maker Samtech one time had an example HDL project code for DMA called “rambo” posted on their website . But after the company got bought out, they took it down. But I suspect spec board makers have the code, and you may ask around for it .

2 Likes

Hi @fvaga and @dlamprid,

Thank you very much for the answer. Our goal is to read the ADC data acquisition that comes from Fmc-adc-iso-400k18b4cha via DDR memory. So, basically the ADC data is written at wb0 interface of ddr3_ctrl and the host can read these data via DMA interface of gn4124_core, which is connect to wb1 interface of ddr3_ctrl.

em_fpga

I would like to make some questions:

  1. SPEC card and SPEC board version 4 are the same board?

  2. How do you have high throughput ADC data transferring from FMC to host via PCI express? Is it via DDR memory?

I am trying to perform this ADC data acquisition by reading data from DDR memory, but I don’t have much information how to do it. I have made some tries using spec.specWriteL comands to setup the DMA controller but I haven’t succeeded. For example, if I try to write 32 bit data in the DDR memory, the DMA controller state machine (dma_controller.vhd) get stuck at the DMA_transfer machine state, waiting for dma_control_done_i = ‘1’. dma_control_done_i comes from p2l_dma_master.vhd and it is asserted to ‘1’ when the condition below is satisfied, having dma_ctrl_done_t <= ‘1’;

          elsif (pd_pdm_master_cpld_i = '1' and pd_pdm_data_last_i = '1'
             and p2l_data_cnt <= 1) then

The commands below shows how I tried to write 32 bit data to the DDR memory in the address 0x00000004:

Setting the DMA controller:
spec.specWriteL(0x4014,0x00000004)
spec.specWriteL(0x4008,0x00000004)
spec.specWriteL(0x4020,0x00000003)
spec.specWriteL(0x4000,0x00000001)

Sending a data to DMA interface (note: I needed to chose an address to be according to the writing command, So I chose the length address)
spec.specWriteL(0x4014,0x11223344)

Disable the DMA transfer
spec.specWriteL(0x4000,0x00000000)

This chipscope picture was triggered when the spec.specWriteL(0x4000,0x00000001) command was send to FPGA.
dma_controller

Best regards,

Orlando.

Hi Orlando,

Version 4 (either 4.0 or 4.1, the differences are mechanical only) is just the latest version of the SPEC. See also here:

In the case of FMC ADC 100M, we do indeed transfer the ADC data first to the DDR memory and then, once the acquisition is complete, we move them with DMA from the DDR memory to the host via PCI express.

Regarding the issues that you are facing, why don’t you try first to read from DDR instead of writing? As I told you before, there is a very simple testbench together with the GN4124 core which does some basic reading over DMA. You can use that as inspiration. You can find it here:

Another thing, if I understand correctly your example, I do not see why you issue the last command (the disable DMA transfer). The DMA start bit auto-clears so there is no need to do it manually (if that’s what you’re doing).

One last note: I just pushed today some fixes and improvements to both the gn4124 core as well as to general-cores. You can now find them both in their respective master branches. I suggest you use these for your tests.

1 Like

Hi, @dlamprid, thank you very much for the information. I remember very much when you’ve recommended me to run the testbench, but we are still waiting to get the Quest/Modelsim license to be able to run it. We expect to have this license in few weeks. Now we just have Xilinx ISIM. Anyway, while I don’t have the Modelsim license, I can investigate the tesbench to understand how it works.

So I will point to master branch for gn4124_core and general-cores. I was pointing to your branch. And I will follow your recommendations by trying reading first and by not switching of the DMA transfer.

I would like to make one more question:

  1. From the software side, do we need to design an user space software (resource4) to manage the data transfering from SPEC DDR memory to host DDR memory or it can be performed just by using spec.specWriteL commands?

Thank you again for all the information!!

Hi, @dlamprid,

I’ve just pointed both gn4124 core and general-cores libraries to master branches.

However, there is a synthesis error. It seems that gn4124_core is instantiating cmp_dma_irq_sync : pulse_synchronizer [line 613], but general cores library has gc_pulse_synchronizer.
Can I instantiate gc_pulse_synchronizer instead?

Cheers,

Orlando.

You’ve probably not really updated the gn4124 core to the latest master!

Please make sure that you’ve pulled all the changes from OHWR. For example, this is the current instantiation of cmp_dma_irq_sync:

1 Like

Now, the updates worked! Thank you!

Cheers.