At present, the core SoCs of high-definition H.264 cameras are basically ASICs. As a programmable device that has developed rapidly in recent years, how can FPGAs have a place in the SoC field of high-definition H.264 cameras? This is what our design needs to achieve.
two.Design Features
Compared with ASIC, FPGA is characterized by strong functions, flexible design, upgrade at any time, accumulation of work results, and low NRE, but the chip price is more expensive than ASIC, so it is necessary to find an application field that can achieve price balance. With the rapid development of smart phones and 3G networks, referring to the technical indicators of the first generation of global eyes of telecom operators, the design indicators are directly positioned at a code stream of less than 512kbps and a resolution of 1280x720x25fps. At present, cameras on the market that can achieve these indicators are almost No, this is a good opportunity for FPGAs.
The stream limit of 512kbps is mainly due to the uploading capability of ADSL and home optical transceivers, and it can also use 3G to upload videos, which reduces the stream download pressure of smartphones; 1280×720 is the mainstream resolution of smartphones, so it is natural to need corresponding Video source, if it shows D1 or CIF, the effect is definitely not ideal.
Our goal is to make a reference design that can be directly used in production. In addition to advanced design indicators, we also need to consider price and practicality, so we chose the low-cost CYCLONE IV series, and also realized all the functions of general network cameras. Such as H.264 compression, 720p25 frame three-stream, two-way voice, definition of important areas and privacy areas, intelligent analysis, motion detection, sound detection, POE power supply, etc.;
three.The key to realizing high-definition low-bit-rate H.264 camera SoC
Generally speaking, the code stream of 1280×720×25fps is about 2Mbps. How to reduce the code stream? In addition to using the H.264 encoder of mail profile with cabac, it is also necessary to analyze the video image, and to balance the frame rate, resolution, video quality, etc., to obtain the best video effect under the 512kbps code stream, In order to achieve such a purpose, complex and flexible control is required, and FPGA can meet such requirements.
Four.Structure of HD Proxy H.264 Camera
The structure diagram of the high-definition low-code H.264 camera is as follows:
1. FPGA selection
In the structure of the high-definition low-code H.264 camera, in addition to the image sensor, memory, power supply and analog interface, all functions such as image processing, H.264 encoding, protocol processing, parameter management, media stream transmission, etc. are performed by FPGA Realization, FPGA is the SoC of high-definition low-bit-rate H.264 camera; at present, there are only a few FPGA manufacturers in the world, and there are not many products to choose from. To use FPGA to realize the SoC of high-definition low-bit-rate H.264 camera, choose A suitable FPGA is very important. The choice of FPGA mainly considers three factors: speed, cost and structure; each FPGA manufacturer has multiple FPGA series, and each series has different speed, performance and price. The cost of the low-bit-rate H.264 camera SoC is as low as possible, and the series with the lowest cost should be selected. For example, the FPGA we chose is the CYCLONE IV series of Altera Corporation, the model is EP4CE115F23C8. The CYCLONE IV is a low-cost series, which is also low-cost. Among the series of FPGAs, the speed of CYCLONE IV is the fastest; the structure of the FPGA is also very important for image processing, especially the number of memory and multipliers. The H.264 algorithm is based on macroblocks, and will inevitably be Involving the input, output and buffering of macroblocks, the data of one macroblock is 384 bytes (256 bytes of luminance data and 128 bytes of chrominance data). That is, 768 bytes. It is just right to use 1 M9K memory block of CYCLONE IV. From the above analysis, it can be seen that the FPGA suitable for image processing requires a small memory block capacity (such as 1KB) and a large number of memory blocks. In addition, The requirements for multipliers are also large; when we choose FPGA, CYCLONE IV has the largest memory logic ratio and multiplier logic ratio among all low-cost FPGAs, which is very suitable for image processing.
2.Image Sensor Selection
In order to meet the requirements of 1280x720x25fps, we only need to choose a megapixel sensor. We finally chose AR0331 with 3 million pixels, which can also achieve 1920x1088x11fps, and Electronic PTZ function when realizing 1280x720x25fps, AR0331 also has wide dynamic function. It is convenient for the camera to expand the wide dynamic function in the future.
3.external memory
The external memory has 2 pieces of LP DDR SDRAM and 1 piece of serial FLASH, the serial FLASH is used to store the configuration data of FPGA and the data program of NIOS, 1 piece of LP DDR SDRAM is used to store image data and coding results, 1 piece of LP DDR SDRAM Run ucLinux for NIOSII.
4. FPGA-implemented functions
The functions implemented by the FPGA as the SoC of the HD H.264 camera are as follows:
① Image processing
● Edge detection RAW interpolation calculation
Simple RAW interpolation calculation will cause edge blur, adding edge detection judgment can avoid edge blur;
● 3D filtering
The image from the image sensor contains noise, and denoising is performed both between frames and within frames;
● 2D sharpening
Due to factors such as the lens, the image from the image sensor needs to be sharpened to be clear;
● GAMMA correction
The grayscale of the image from the image sensor is incorrect, and GAMMA correction is required to obtain the correct grayscale;
● Automatic exposure control
Adjust the appropriate exposure parameters according to the brightness conditions;
● OSD Display
2048-bit images can be superimposed, the images are in macroblocks, and the structure can be selected from 128×1, 64×2, 32×4, 16×8, 8×16, 4×32, 2×64, 1×128, etc. The position of the screen can be set;
● Regional management
Privacy area, motion detection area, etc. can be defined;
② Encoding processing
● Encoding format
Encoding with the H.264 main profile with cabac algorithm can improve the compression rate and reduce the code stream; since the full implementation of all H.264 algorithms requires a lot of resources and a large-capacity FPGA, this is unrealistic. 1 test sequence has been tested, and the compression rate of 2 reference frames is 5% higher than that of 1 reference frame, but the processing of 2 reference frames requires far more than 5% more resources than the processing of 1 reference frame. For We do not use some methods that are very complicated and do not improve the compression rate for the time being. Reducing the code stream is a systematic task and needs to be considered from multiple aspects; in addition, we must ensure the accuracy of the algorithm to avoid errors at the decoding end;
● Coding ability
The maximum is 1280x720x25fps. If you need to encode multiple code streams at the same time, you need to consider that the sum of the encoding numbers of macroblocks per second of the three code streams cannot be greater than 90000, such as 1280x720x23fps+320x180x23fps, 1136x640x24fps+568x320x24fps+284x160x24fps, etc.;
● Image quality control
Coding and quantization parameter adjustment range: 24~41, small value, large code stream, good image quality, large value, small code stream, poor image quality;
● Stream control
The code stream control mode is CBR, which controls the average code stream;
③ Network processing
● Peak network speed
At present, the network interface uses full-duplex 100M Ethernet. Since the amount of encoded data of the image is uneven, the amount of encoded data of the I frame is large, and the amount of encoded data of the P frame is small, so the amount of data is pulsed. When encoding data, try to use a large network speed to reduce the delay, but you should consider the affordability of ADSL, and you can set a network speed peak of 1~70M;
● Stream mean
The average code stream is the amount of data transmitted in 1 second, and the average code stream can be set from 16Kbps to 8128Kbps; the code stream control method of CBR is based on the average code stream;
● Stream buffer
Since the amount of coded data is distributed in pulses, when the coded data generation speed is higher than the peak network speed, the coded data needs to be cached in the LP DDR SDRAM, otherwise it will be lost;
● Protocol
TCP, RTP, RTSP, DHCP, NTP, HTTP, FTP, SMTP and DDNS;
④ Management
● NIOS II
NIOS II is a built-in CPU, running ucLinux, which is responsible for the buffering and packet sending of media stream data;
● Parameters (provided via WEB)
Receive parameter setting and update, keep 2 sets of parameters, 1 copy is the factory parameter, 1 copy is the parameter actually used, the parameter actually used during normal power-on takes effect, and the factory parameter takes effect when the reset button is pressed;
● Upgrade (provided via WEB)
There are two configuration data stored in FLASH, one is the configuration data when leaving the factory, and the other is the configuration data after the upgrade. data loading;
● Alarm interface (provided through WEB)
Provide upload function of alarm signal;
⑤ Audio
Support two-way 64kbits ALaw and 16kbits/32kbits ADPCM transceiver, sound detection.
five.concluding remarks
The high-definition low-bit-rate H.264 camera using FPGA as SoC has been designed and achieved the design goal. The video quality is clear and smooth, which fully meets the requirements of smartphones as monitoring terminals.
The Links: NL6448BC20-08E 7MBR100VX120-50
