WRTD installation instructions

Finally, following the v1.0.0 release, and while we are still waiting for RPM packages, we have made available build scripts that will greatly simplify the process of installing WRTD (at least on CentOS7).

Please head over to the installation instructions wiki page for more details.

Keep in mind that our drivers still do not support MEN A25 VME CPUs, we are working on this and will provide you with an update via this forum as soon as the new drivers become available.

Don’t forget to submit issues if you encounter problems/bugs!

The issue with the MEN A25 has been addressed and the installation instructions have been updated accordingly.

In CentOS7, there is still a warning produced by the Linux kernel when the drivers are first loaded, probably due to a spurious interrupt. We are investigating the source of this issue, it’s probably related to the FPGA-based implementation of the VME bridge on the MEN A25. Still, it’s just a warning and it does not affect the operation of the WRTD SVEC-TDC-FD reference Node.

As always, don’t forget to submit issues if you encounter problems/bugs!

Dear Dimitris Lampridis,

We had alreadly begun our project about the WRTD. However, we just met a problem when we wanted to do the FPGA Programming.

To complete this step, we had downloaded the latest binaries and prepared to flash the FPGA bitstreams permanently to the Nodes via JTAG. We also found a related reference which called svec-gateware-manual.pdf (https://www.ohwr.org/project/svec/wikis/Documents/SVEC-Gateware-Manual).

According to this file, under Linux, we found we had to use a lot of software files from the software subdirectory in the SVEC project’s repository (http://www.ohwr.org/projects/svec/repository/); nevertheless, we cannot log into this website which needed username and passward. Is there any other method can get the files of the software subdirectory in the SVEC project’s repository under Linux?

Hence, we tried to go these peocesses under window10. When we wanted to update the bootloader through JTAG in ISE14.7, we got Programming Failed after we click the program button of the Flash following the reference—svec-gateware-manual. We also can transfer the bit file of the latest binaries (wrtd-binaries-v1.0.0) to MCS file; however, when we updated the MCS file and wanted to program the FPGA, the result is Failed. Could you give us any suggestion in the processes of FPGA programming or tell us any other useful methods to solve the problem we met?

Thanks,
Pengxiang Yu
SSRF | SXFEL | SHINE, China

Hi,

I have just updated the installation scripts to also build and install the svec-flasher tool. So if you fetch the latest commit from the master branch of the https://gitlab.cern.ch/dlamprid/ohwr-build-scripts.git repository and re-run the wrtd_ref_svec_tdc_fd_install.sh installation script, you should now get the svec-flasher tool installed under /usr/local/bin at the end of the installation.

From there, you can use the tool to flash the bitstream that you downloaded, following the instructions from the SVEC gateware manual, section 4.1.

It could be though that you have messed up the bootloader when you tried to write directly to the flash memory. In that case, you can get the v3 bootloader bitstream (for the small “system” FPGA of the SVEC) from here and flash it using the instructions from the SVEC gateware manual, section 4.4. Once the system FPGA is programmed, you can use the svec-flasher tool to flash the WRTD bitstream.

I have also updated the installation instructions with the above information:

Dear Dimitris Lampridis,

Thank you for your suggestions in last email. According to your email, we had successfully gotten the svec-flasher tool under the /usr/local/bin directory. However, when we wanted to flash the bitstream to the FPGA following the steps you update in the website(https://www.ohwr.org/project/wrtd/wikis/Installation-Instructions#flashing-the-svec-fpga), we met some problems.

Firstly, we just follow the instructions provided in section 4.1 of the SVEC Gateware Manual, we got:
svec-flasher 3 wrtd_ref_svec_tdc_fd.bin
Bootloader version: 3
Programming the Application FPGA flash with bitstream wrtd_ref_svec_tdc_fd.bin.
Flash memory ID invalid. You are probably be running an old version
of the bootloader that doesn’t support flash programming via VME.

Hence, we thought maybe the bootloader of my SVEC(SVEC v3) is corrupt which you also mention in website, we tried to update the bootloader through VME following the instructions provided in section 4.3 of the SVEC Gateware Manual, we got the same problem:
svec-flasher 3 svec-bootloader-v3-20140815.bin -b
Bootloader version: 3
Programming the Application FPGA flash with bitstream svec-bootloader-v3-20140815.bin.
Flash memory ID invalid. You are probably be running an old version
of the bootloader that doesn’t support flash programming via VME.

Then, we wanted to update the bootloader through JTAG following the section 4.4. When we tired to do the last step: Right click on the “FLASH” chip above the “xc6slx9” and select “Program”. Select the“Verify” option and click OK, the result showed “Program Failed”. The Console showed the error:
PROGRESS_START - Starting Operation.
INFO:iMPACT:583 - ‘2’: The idcode read from the device does not match the idcode in the bsdl File.
INFO:iMPACT:1578 - ‘2’: Device IDCODE : 00000000000000000000000000100010
INFO:iMPACT:1579 - ‘2’: Expected IDCODE: 00000100000000000001000010010011
PROGRESS_END - End Operation.
Elapsed time = 0 sec.

However, if we did not select the “Verify”(also use the Xilinx Platform Cable USE Model DLC9LP through ISE14.7 IMPACT), it showed program succeeded. When we tried to program Application FPGA Flash through VME follow the instructions provided in section 4.1. We got the same problem which is :
Bootloader version: 3
Programming the Application FPGA flash with bitstream wrtd_ref_svec_tdc_fd.bin.
Flash memory ID invalid. You are probably be running an old version
of the bootloader that doesn’t support flash programming via VME.

Could you please find the problems for us according to the errors I post above? Or is there any suggestion can help us solve the problems?

Thanks,
Pengxiang Yu
SSRF | SXFEL | SHINE, China

The IDCODE reported by Impact does not belong to any Xilinx Spartan-6 FPGA (see Table 5-13 of the Spartan-6 FPGA configuration guide, UG380).

This sounds to me like a hardware issue. Did you try with more than one SVEC? Another JTAG programmer and/or different cables?

Dear Dimitris Lampridis,

According to your suggestion in last email, we tried to do the same steps in another SVEC board(opertaing environment: Window10, ISE14.7, and Xilinx Platform Cable USB Model DLC9LP). We got the same problem of the last SVEC board:
PROGRESS_START - Starting Operation.
‘2’: SPI access core not detected. SPI access core will be downloaded to the device to enable operations.
INFO:iMPACT - Downloading core file C:/Xilinx/14.7/ISE_DS/ISE/spartan6/data/xc6slx9_spi.cor.
‘2’: Downloading core…
LCK_cycle = NoWait.
LCK cycle: NoWait
done.
‘2’: Reading status register contents…
INFO:iMPACT:2219 - Status register values:
INFO:iMPACT - 0011 1100 1110 1100
INFO:iMPACT:2492 - ‘2’: Completed downloading core to device.
‘2’: ID Check passed.
‘2’: ID Check passed.
‘2’: Erasing Device.
‘2’: Using Sector Erase.
‘2’: Programming Flash.
‘2’: Reading device contents…
Failed at address, 340736
‘2’: Verification Terminated
INFO:iMPACT - ‘2’: Flash was not programmed successfully.
PROGRESS_END - End Operation.
Elapsed time = 28 sec.

When we used another JTAG cable(JTAG-HS3 Rev.A), we got:
INFO:iMPACT:2219 - Status register values:
INFO:iMPACT - 0000 0000 0010 0010
INFO:iMPACT:2492 - ‘2’: Completed downloading core to device.
INFO:iMPACT - ‘2’: Flash was not programmed successfully.
PROGRESS_END - End Operation.
Elapsed time = 0 sec.

Could you please give us some suggestion to solve the problem we found or What is the reasons cause the problems.? What’s more, could you tell us the opreating environment you had including the opreating system, JTAG programmer, and cables?

Thanks,
Pengxiang Yu
SSRF | SXFEL | SHINE, China

Hi,

Well, that is not the same error message as before. It looks like the IDCODE is correct but it fails while reading back the device contents, at some offset.

To be honest, I don’t see what else could be wrong other than your setup somehow.

The SVEC is a popular board, deployed in many places, it does not require any special operating environment to program it.

Still, for completeness, I personally use Xilinx ISE 14.7 on Debian Linux 10, with these drivers:
https://rmdir.de/~michael/xilinx/

The JTAG programming cable is a XILINX Platform Cable USB II.

Dear Dimitris Lampridis,

This time we did the same steps in an new opertaing environment: Window10/window7, ISE14.7, and Xilinx Platform Cable USB II Model DLC9LP. We got the problem which was same as the last time.

Also, we tried the operating environment you mentiond: Xilinx ISE 14.7 on 64bit Debian Linux 9 , with the drivers(https://rmdir.de/~michael/xilinx/) and JTAG programming cable (XILINX Platform Cable USB II DLC10). We got the same result as last time. The result is:
Selected part: M25P128
Unprotect sectors: FALSE
INFO:iMPACT - A CFI file is not detected. To ensure correct and safe
configuration,
Please make sure a CFI file is present in the same directory as the
PROM file,
or, regenerate the PROM file with the latest software.
INFO:iMPACT - Current time: 12/18/19 9:00 AM
PROGRESS_START - Starting Operation.
Maximum TCK operating frequency for this device chain: 25000000.
Validating chain…
Boundary-scan chain validated successfully.
‘2’: IDCODE is ‘20ba18’ (in hex).
‘2’: ID Check failed.
INFO:iMPACT:2488 - The operation did not complete successfully.
INFO:iMPACT - SPI Device not found.
INFO:iMPACT:2488 - The operation did not complete successfully.
INFO:iMPACT - ‘2’: Flash was not programmed successfully.
PROGRESS_END - End Operation.
Elapsed time = 0 sec.

We found when we want to upload the svec-bootloader-v3-20140815.mcs, the console of the IMPACT shows A CFI file is not detected. We cannot find the svec-bootloader-v3-20140815.cfi directly so that we try to create a mcs file from svec-bootloader-v3-20140815.bit. However, the result was same as above(but the CFI can be detected this time).

Is that you use the mcs file which is created by yourself rather than download from website (https://www.ohwr.org/project/svec/wikis/Gateware-Release-3-0)?

Thanks,
Pengxiang Yu
SSRF | SXFEL | SHINE, China

Dear Dimitris Lampridis,

As to the questions we discussed before, we have finally identified the problems through our efforts:

The SVEC board we bought which was manufactured in 2018; however, we found the manufacturer had already changed the flash of the SVEC board in Dec, 2017.(https://www.ohwr.org/project/svec/issues/10) Hence, we cannot update the bootloader through JTAG according to the section 4.4 of the SVEC Gateware Manual. After we changed the flash type from “M25P128” to “N25Q128”, the programming succeed. (https://forums.xilinx.com/t5/Vivado-Debug-and-Power/iMPACT-SPI-Flash-Programming-Support/td-p/874034)

Now come back to the installation instruction, we prepare to use the svec-flasher tool to flash the bitstream to the FPGA(section 4.1 of the SVEC Gateware Manual). However, we got an same error with the same result of running the command of section 4.3:
Bootloader version: 3
Programming the Application FPGA flash with bitstream wrtd_ref_svec_tdc_fd.bin.
Flash memory ID invalid. You are probably be running an old version
of the bootloader that doesn’t support flash programming via VME

When we searched the error in tbe source code of the svec-flasher, we found the problem came from the line 28 and 294-298 of svec-flasher.c , in which the code still use “M25P128”.
(https:// ohwr. org/project/svec/blob/master/software/tools/vme-flasher/svec-flasher.c)

Could you please just change the code for us, or could you give us some suggestions about where and how should we modify the code?

Thanks,
Pengxiang Yu
SSRF | SXFEL | SHINE, China

Dear Dimitris Lampridis,

Happy New Year!

Firstly, we just want to report our lastest progress. As to the last questions we sent to you, we compared the manual of the old flash M25P12 (https://media.digikey.com/pdf/Data%20Sheets/Micron%20Technology%20Inc%20PDFs/M25P128_Rev_A.pdf) with the manual of the new flash N25Q12 (https://www.micron.com/-/media/client/global/documents/products/data-sheet/nor-flash/serial-nor/mt25q/die-rev-a/mt25q_qlhs_l_128_aba_0.pdf). We found we might make the FPGA programming succeed by changing some lines of the svec-flasher.c. Therefore, we added some commands into the svec_carrier_build.sh which I showed below:
sed -i ‘s/0x202018/0x20ba18/’ software/tools/vme-flasher/svec-flasher.c
sed -i ‘s/0x40000/0x10000/’ software/tools/vme-flasher/svec-flasher.c
Then, after we reexecute the commands bash wrtd_ref_svec_tdc_fd_install.sh and svec-flasher 3 wrtd_ref_svec_tdc_fd.bin, the results showed the programming succeed:
[root@localhost gateware]# svec-flasher 3 wrtd_ref_svec_tdc_fd.bin
Bootloader version: 3
Programming the Application FPGA flash with bitstream wrtd_ref_svec_tdc_fd.bin.
Programming page 0/0.
Verification…
Programming page 16506/16506.
Verification…
Programming OK.
Are there any problems in our process? We thought it worked.

Secondly, we used svec-flasher tool to program the wrtd_ref_svec_tdc_fd.bin successfully. About wrtd-rt-tdc.bin or wrtd-rt-fd.bin of firmware file, do we need to used svec-flasher tool again or load them in runtime? Also, where should we program with bitstream used these bin files?

Last but not least, about VME registration of the Installation Instruction, how can we make sure the address and irq of different slots? We also found there was a similar question (How do I program the relocation registers of the SVEC to set up its base address?) that posted in FAQ (https:// www.ohwr. org/project/svec/wikis/faq). According to the contents of this FAQ, there was a vme64x_user_manual.pdf(https:// www. ohwr. org/project/vme64x-core/blob/master/documentation/user_guides/vme64x_user_manual.pdf); however, we could not access it. Are there any other documentations we can refer to?

(Sorry, because the new user can only put 2 links in a post, I break the last two hyperlink address)

Thanks,
Pengxiang Yu
SSRF | SXFEL | SHINE, China

Hi again, and happy new year!

I’m glad you managed to solve the programming issue. I was not aware of the fact that the new flash on the SVEC is not fully compatible with the old one. I thought we had replaced it with something compatible. I’ll discuss this with my colleagues (@Erik.van.der.Bij, @tristan.gingold) and provide a more permanent solution.

Regarding your remaining questions:

  1. Yes, it looks like your programming finally succeeded!
  2. There is no need to program the firmware files, they are already included in the bitstream you used to program the FPGA. If however your ever need to do so (e.g. if you want to change the firmware to your own), you can use the mockturtle-loader utility that was already installed by the installation scripts.
  3. Regarding the VME registration, I know that my software colleagues (@fvaga) are working on a more plug-n-play approach, but until this is ready, we have to use a more manual approach.

All in all, you should be ready to use the SVEC now.

For future issues, please start a new thread, to make it easier for other users to search and find answers to their questions.

I’m sorry about the problems you have had with accessing the repository. We have changed the ohwr server and the new addressing scheme has become different.
The repositories are now available under
https://ohwr.org/project/svec/tree/master (and not under svec/repository anymore).
As the information is Open, all information on ohwr.org should be available without any password :-)

Best regards,
Erik
CERN

Hello,

As to the third question about the VME registration, if we want to use the manual approach, how can we make sure the base address and irq vector corresponding to different slots?

Thanks,
Pengxiang Yu
SSRF | SXFEL | SHINE, China

You can use anything you want for address and IRQ vector, they don’t need to correspond to anything physical (such as slot number), as long as they don’t collide with each other (in which case, the tool should anyway complain).