Git Product home page Git Product logo

circuitvalley / usb_c_industrial_camera_fpga_usb3 Goto Github PK

View Code? Open in Web Editor NEW
780.0 41.0 145.0 58.75 MB

Source and Documentation files for USB C Industrial Camera Project, This repo contains PCB boards, FPGA , Camera and USB along with FPGA Firmware and USB Controller Firmware source.

C 12.93% Assembly 0.04% Makefile 0.05% Verilog 86.98%
camera fpga usb usb3 verilog csi mipi mipi-csi-receiver uvc

usb_c_industrial_camera_fpga_usb3's Introduction

USB C industrial camera with Interchangeable C mount Lens, Interchangeable MIPI Sensor with Lattice Crosslink NX FPGA Cypress FX3 USB 3.0 controller

FPGA ISP Pipeline Specifications

MPI CSI receiver ISP has No virtual restriction on supported frame rate or resolution. Tested more than 4K with IMX477 4056x3040. Can do 8K with around 30FPS or even higher than that as long as FPGA is fast enough for needed frame rate and FPGA/Board has enough memory to be able to store minimum 4 Line worth of pixels. Output Clock is independent of MIPI clock. Easily Portable code to Xilinx or any other FPGA, No Vendor specific components has been used except for the PHY itself which can be replaced by other vendor's DDR phy and Embedded Block RAM. Only Debayer/Demosaic and Output reformatter need Block RAM. Block ram can also be replaced vendor's RAM. Auto detection of RAW pixel width supporting different camera sensors and sensor modes without FPGA reconfiguration.

Speed

  • Supports MIPI bus clock 900Mbitsps Per lane with upto 4 Lanes, Total 3.6Gbitsps Sensor bit stream, Has been Tested upto 900Mbitsps with 8x Gear.
  • Pixel Processing pipeline with 2,4 or 8 Pixel per clock can reach more than 110Mhz with Lattice Crosslink-NX LIFCL-40 High Speed, So basically Can process upto 880 MegaPixels per second. With this can reach Around 120FPS with 4K resolution and around 30 FPS with 8K. Or even 3000 FPS with 640 x 480 as long as Camera and MIPI Wire allows. With Different Faster FPGA speed will be more.
  • FPGA Oputput Pipeline is decoupled from MIPI clock, runs on output clock, It feeds into Cypress FX3 32bit GPIF can do Max 160Mhz. Cyress FX3's specs limits max GPIF clock to 100Mhz.

Configurability

  • Selectable max RAW pixel width
    FPGA Design is configurable with parameters to support pixel depth from RAW10 to RAW14 or Virtually any bit depth even 16bit RAW when it becomes a MIPI Specs. Parameter specify maximum pixel width that is supported while module auto detect package type at runtime with RAW14 selected as max pixel width, RAW10, RAW12 and RAW14 will be automatically detected and processed
  • Selectable number of MIPI lanes
    With just definition of Parameter value number of lane is also configurable between 2 or 4 MIPI lanes.
  • Selectable Pipeline Size
    Pipeline is Configurable with a parameter to Process 2,4 or 8 Pixel. 2 Pixel Per Clock is only available with 2 Lane MIPI, while 8 Pixel Per Clock is only available with 4 Lanes.
  • Selectable MIPI Gear Ratio
    User can select weather to operate MIPI/DDR phy in 16x or 8x Gear ratio. Most DDR/MIPI phy supports 8x Gear while few do support 16x gear.
    Block RAM and DDR PHY IPs need to be manually regenerated if Gear, pixel width , lane or PPC is changed.
  • Selectable MIPI continuous clock mode
    User and select between MIPI clock lp based Frame sync or Frame start and frame stop packt based frame sync. Some MIPI cameras do not support going to LP mode while frame blank occur, With this option user can enable Frame Start and Frame stop detection, to have a frame sync.
  • Selectable ROM based Sample Generator
    For ISP debuging ROM based sample generator can be activated. Two ROM lines are there have both even and odd line to full image test.

Tests

IMX477

  • 4 Lane 12 bit IMX477
    4056x3040 15 FPS Full Sensor
    2028x1520 60 FPS Full Sensor Binned 2x2
    2028x1080 100 FPS

  • 4 Lane 10 bit IMX477
    1332x990 150 FPS Binned 4x4
    640x480 400 FPS Binned 4x4
    640x128 2400 FPS Binned 4x4

  • 2 Lane 12 bit IMX477
    4056x3040 10 FPS Full Sensor
    2028x1520 35 FPS Full Sensor Binned 2x2
    2028x1080 50 FPS

  • 2 Lane 10 bit IMX477
    1332x990 100 FPS Binned 4x4
    640x480 200 FPS Binned 4x4
    640x128 1200 FPS Binned 4x4

IMX219

  • 4 Lane 10 bit IMX219
    3280x2464 30 FPS
    1280x720 240 FPS
    1920x1080 120 FPS
    640x480 400 FPS
    640x128 1300 FPS
    640x80 2000 FPS

  • 2 Lane 10 bit IMX219
    3280x2464 15 FPS
    1280x720 60 FPS
    1280x720 120 FPS
    1920x1080 60 FPS
    640x480 30 FPS
    640x480 200 FPS
    640x128 600 FPS
    640x80 900 FPS

IMX290/IMX327/IMX462

  • 4 Lane 12 bit
    1280x720 120 FPS
    1920x1080 120 FPS

  • 2 Lane 12 bit
    1280x720 60 FPS
    1920x1080 60 FPS

IMX219

Project Blog post

https://www.circuitvalley.com/2022/06/pensource-usb-c-industrial-camera-c-mount-fpga-imx-mipi-usb-3-crosslinknx.html

Shield: CC BY 4.0

This work is licensed under a Creative Commons Attribution 4.0 International License.

CC BY 4.0

alt text

alt text

alt text

alt text

alt text

alt text

usb_c_industrial_camera_fpga_usb3's People

Contributors

circuitvalley avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

usb_c_industrial_camera_fpga_usb3's Issues

GigE Expected???

A couple of more things to address:

  • IMX296 support expected
  • GigE expected
  • FPGA selection: Why did you select LIFCL-40-9BG256C? Maybe LIFCL-40-9BG256C is good for IMX219, but what about IMX296? which one is more suitable? Maybe ECP2/M ?? Hmmm... A little complicated to select a suitable FPGA.

resistor values in BOM are incorrect

The resistor values in all the 3 BOM files are incorrect.

For example, in \Hardware\01_Camera_Sensors\IMX290_IMX327_IMX462\IMX290_Module_BOM.xlsx
image
All the four resistors are annotated as "0 ohm", which doesn't match with the schematic:
image
That's also the case in FPGA_bom and usbc_bom, causing confusion during the assembly.

Please change the Comment field of the resistors from "Thick Film" to "=Value", and update all three BOM files.
image

Files for debayer simulation

Hello,
can you upload the files used in the debayer testbench such as
italy.bmp768x512.raw
italy.bmp768x512.raw.rgb
italy.bmp768x512.raw.yuv
italy.bmp768x512.raw.verify

Has anyone successfully reproduced this project?

Has anyone successfully reproduced this project? I've been struggling with this project for a long time without success.

The FPGA is programmed, and the fx3 runs successfully (the PC can recognize the usb device), but the image cannot be obtained using webcamod8.8.

I guess there are two reasons: 1. There is a problem with the driver configuration of fx3 and imx219 or imx477. 2. There is a problem with the fpga configuration code.

Hope that someone who has successfully copied can provide suggestions or comments, thank you

Version firmware ENABLE and pinout

Hi @circuitvalley thanks for your design, it's nice.

I want understand your design and need resolve any questions. I hope your help.

I'm going to use your design with sensor IMX290 but I have watched a change no implemented at pin ENABLE, this pin active XCLR from delay ENABLE in PCB IMX477 but PCB IMX290 active from FPGA. In your firmware, where can I watch this change?

In your sketch FPGA I he watched a pinout different respect your schematic, is possible different version? For example, J12 is DATA_0 but in schematic J12 is D8, it's showed in the next figure:

Captura de pantalla 2023-07-03 125154
Captura de pantalla 2023-07-03 125231

Thanks for your time,
Rubén

ECP5 supported?

Hello,
Amazing project. I was wondering if it is possible to use this with an ECP5 FPGA?

Anyone with extra hardware?

I'd like to use this project as a development/learning platform but I don't think it's practical for me to get a single piece of each board manufactured for myself - Does anyone have any extra hardware that they'd be willing to sell to someone in the US?

Thanks

Faild to use HW-USBN-2B tool to programming the FPGA

Hi @circuitvalley, I am trying to replicate this great project, but currently I am encountering a burning issue with the FPGA.

I am using the HW-USBN-2B programmer to program the external FLASH, the connection shown in the figure (GND, JTAG-EN, TMS, TDI, TDO, TCK).
微信图片_20230522213934
But the programming software always reports an error and cannot read the Device's IDCODE. It say Failed to read the Device's IDCODE.
微信图片_20230522213925
By checked the hardware connection and found no clue. Please give me suggestions and comments if it is convenient,I appreciate your help!!

some schematic is missing

In the FPGA_core project,two 40 pin dense connectors J1 and J2 appear on the PCB drawing,but there's no J1 or J2 on any of the schematic currently published. Please check if files under \Hardware\02_FPGA_Core\PCB_Source is complete

Unable to find the instance/port 'mipi_data_p_in1[1]' in the constraint 'ldc_set_location -site {C1} [get_ports {mipi_data_p_in1[1]}]'

Hi @circuitvalley

Full fpga build log.
fpga-build-log.txt

I still try to repuduected this project, I encountered no signal output from some key pins(like Frame-sync, Line-Sync).
image

I found some clues when checking the compilation information. Does this mean that some signals of mipi are not configured correctly?
image

I am using the default pdc file, please give me some suggestions, thank you.

No ENABLE signal to boot the IMX290 sensor from the lattice FPGA

Hi @circuitvalley,
After program the fpga and usb3, finnaly I got a another issue, the IMX290 sensor board has no power. The ENABLE signal was always low level. I suspect that the fpga program is not normally started from the external flash, but I don't know how to verify this problem.
Selection_048
The Cypress FX3 USB3 serial port output log:
20230523-181254
I think I should be very close to the final success, please help me, Thank you very much!!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.