Example using camera with XIP kernel 4.9

Here's the status on camera interfacing (for RZ/A Linux that is).
* There is a old V4L2 CEU driver in the kernel, but it's got some issues and more work that it's worth
* There is some work going on in the upstream team to make a new V4L2 CEU driver, but it's not clear when that will get done
* I created a very simple example at one point that basically sets up all the CEU and OV7670 registers from a user app (not a kernel driver). Basically, no V4L2 (just read camera frames to a buffer so you can pass them to OpenCV, libJPEG, the LCD, or whatever you want).

Since #1 and #2 are not great options for right now, I was going to clean up #3 and post it. It was done with Linxu-3.14, but as I mentioned, it's all contained in a user app, so there isn't really and "porting" except add the pin mux setup in the Device Tree.

So the question is, do you want the current ugly version now? Or wait till it's cleaned up and posted? (probably December time frame)

NOTE: On the RZ/A1 RSK board, you can't use the LCD and CEU camera at the same time (pin conflict). So you have to save JPEGs to a file on a SD card/USB stick, or stream them via Ethernet/HTTP.

Chris

Thanks for your reply Chris.
I'll receive my hardware in early december (board based on RZ/A1L, single QSPI, OV6211) and my aim is to ensure our hardware can get frame from OV6211. I'm afraid I'll run out of time if I wait for the new driver.
Actually I wanted to use 4.9 kernel to play around but if there is a better solution with the 3.14 I don't mind using it again :)
As long as it works every solution suit me well !

Not yet.
The reason was that with the RZ/A1H RSK board, you can't use the LCD and camera at the same time. Also, with the RZ/A1L Stream-it board, you can't use the SDRAM and camera at the same time.
However, a new version of the Stream it board is coming out that with a resistor change, you can use SDRAM, LCD and camera at the same time. So, that was what I was going to base my example code on.

Just FYI, I cleaned up that camera code and added it to the 4.9 BSP.
github.com/.../6cfd6f6758596b06c6adcb9af4583daf825e2d76

I used a new Stream it V2.3 board that can be easily modified to support the CEU interface and Linux at the same time.

The code supports both OV7670 and OV7740 cameras which came with the Stream It kit.

The app code is pretty generic, and should work with any board (but you have to set up the CEU pins yourself)

Thanks a lot Chris ! I'll give it a try. Currently I'm using the ongoing upstream CEU driver and a 4.15 kernel it works so far with an OV7670 (I'm still working on OV6211)

I've been meaning to try that new CEU driver and backport it to 4.14 (our next BSP kernel version). I wasn't sure if it would back port easily to 4.9, so I just put up the sample code that sets the registers from user space.

Out of curiosity, is everything you need (RZ/A1 drivers) in the 4.15 upstream release?

> Regarding the 4.15 kernel I only had to add and patch AXFS to the kernel.

Can I get your AXFS patch??? It's the only thing I need left before we can release the 4.14 BSP. I've been meaning to get back to it, but haven't.

If it looks good, maybe you can submit a pull request to the official repo ( github.com/.../ ).

> Can I get your AXFS patch???
Sure, I'll open a PR before the end of the day.

I just saw the PR. Thanks!

At first glance, I have some questions on it, but I'll ask them via github.

One more think I forgot to mention was that now as of 4.15, cramfs has an XIP option now. However unlike AXFS, it's all or nothing when you choose to XIP a file (you can't select just the pages you want XIP-ed). However, it is an 'upstream' option. But, you do need a new mkcramfs tool to use it. I plan on updating Buildroot (upstream) at some point (hopefully sooner than later)

Now you're talking about XIP option in upstream kernel, I remember I had to do some dirty hacks to be able to build my kernel as XIP.

Yes, no matter how many times I've tried, and as many different patches I have submitted, the upstream ARM maintainer will not let me enable XIP for MMU enabled systems. (although he is OK with keeping the underlying code working).

So, you always have to apply this simple patch to be able to enable XIP_KERNEL:

diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 7f61372a6462..d33e20ffcbe6 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -331,7 +331,7 @@ config ARCH_MULTIPLATFORM
bool "Allow multiple platforms to be selected"
depends on MMU
select ARM_HAS_SG_CHAIN
- select ARM_PATCH_PHYS_VIRT
+ select ARM_PATCH_PHYS_VIRT if !XIP_KERNEL
select AUTO_ZRELADDR
select TIMER_OF
select COMMON_CLK
@@ -1966,7 +1966,7 @@ endchoice

config XIP_KERNEL
bool "Kernel Execute-In-Place from ROM"
- depends on !ARM_LPAE && !ARCH_MULTIPLATFORM
+ depends on !ARM_LPAE
help
Execute-In-Place allows the kernel to run from non-volatile storage
directly addressable by the CPU, such as NOR flash. This saves RAM


Since 4.16.-rc1 was released yesterday, I'll have to confirm sometime this week that nothing broke for XIP_KERNEL.

Hello!

I'm also trying to use an OV7670 with Linux-4.9, but in a RSK+ (RZA1H). With "ceu_example", the code stucked in the while around "ceu_is_capturing" function (CAPSR register always 0x01). You said we must change CEU pin setup in u-boot or kernel, but I can't figure out where... The ideia is "streaming" the images to a webpage, just like 3.14 example.

Thanks a lot!

Remember, on the RSK board, you can not use camera and LCD at the same time. They share pins.
So, you should remove LCD display section (vdc5) from the Device Tree so the pins do not get configured as LCD.

Or, you can set the status to disabled.
&vdc50 {
pinctrl-names = "default";
pinctrl-0 = <&vdc50_pins>;

display = <&display0>;
status = "disabled";


Then, to set the pins to CEU, you can do this in u-boot, add this code to function board_early_init_f( )
[file] u-boot-2017.05/board/renesas/rskrza1/rskrza1.c

/* Enable CEU Camera interface */
pfc_set_pin_function(10, 1, ALT1, 0, 0); /* VIO_VD */
pfc_set_pin_function(10, 2, ALT1, 0, 0); /* VIO_HD */
pfc_set_pin_function(10, 0, ALT1, 0, 0); /* VIO_CLK */

pfc_set_pin_function(10, 4, ALT1, 0, 0); /* VIO_D0 */
pfc_set_pin_function(10, 5, ALT1, 0, 0); /* VIO_D1 */
pfc_set_pin_function(10, 6, ALT1, 0, 0); /* VIO_D2 */
pfc_set_pin_function(10, 7, ALT1, 0, 0); /* VIO_D3 */

pfc_set_pin_function(10, 8, ALT1, 0, 0); /* VIO_D4 */
pfc_set_pin_function(10, 9, ALT1, 0, 0); /* VIO_D5 */
pfc_set_pin_function(10, 10,ALT1, 0, 0); /* VIO_D6 */
pfc_set_pin_function(10, 11,ALT1, 0, 0); /* VIO_D7 */


Or, instead you can add the pin setup to the Device Tree in the kernel.
The easiest is to just add the CEU pins to a driver that you already know will be used.
For example, the serial console. Then, when the serial console is loaded, the CEU pins will also get set up.
Just remember that you have to remove the ";" in the list (after RxD2) and and replace it with a ","

Something like this:
[FILE] linux-4.9/arch/arm/boot/dts/r7s72100-rskrza1.dts

&pinctrl {

/* Serial Console */
scif2_pins: serial2 {
pinmux = <RZA1_PINMUX(3, 0, 6)>, /* TxD2 */
- <RZA1_PINMUX(3, 2, 4)>; /* RxD2 */
+ <RZA1_PINMUX(3, 2, 4)>, /* RxD2 */
+
+ /* Enable CEU Camera interface */
+ <RZA1_PINMUX(10, 1, 1>, /* VIO_VD */
+ <RZA1_PINMUX(10, 2, 1>, /* VIO_HD */
+ <RZA1_PINMUX(10, 0, 1>, /* VIO_CLK */
+
+ <RZA1_PINMUX(10, 4, 1>, /* VIO_D0 */
+ <RZA1_PINMUX(10, 5, 1>, /* VIO_D1 */
+ <RZA1_PINMUX(10, 6, 1>, /* VIO_D2 */
+ <RZA1_PINMUX(10, 7, 1>, /* VIO_D3 */
+
+ <RZA1_PINMUX(10, 8, 1>, /* VIO_D4 */
+ <RZA1_PINMUX(10, 9, 1>, /* VIO_D5 */
+ <RZA1_PINMUX(10, 10,1>, /* VIO_D6 */
+ <RZA1_PINMUX(10, 11,1>; /* VIO_D7 */
+
}


That should do it.

Hi Chris, thanks for your help!

I've disabled "vdc50" and added "PINMUX" on Device Tree, but no success... App still stucking on "ceu_is_capturing"... I've also tried "pfc_set_pin_function", with no luck... Double checked the connections, they're fine too!

Thanks again

Don't you have some soldering to do (moving some resistors) to be able to use the camera on this board ? I never used this board.

First make sure something is coming out of the camera pins. That will confirm that the I2C connection is working.

Even if you set up the CEU pins correctly, if the CEU never sees a VSYNC, HSYNC, or clock, it will never trigger.

First make sure something is coming out of the camera pins. That will confirm that the I2C connection is working.

Even if you set up the CEU pins correctly, if the CEU never sees a VSYNC, HSYNC, or clock, it will never trigger.

Hello guys

I only had to solder a connector for the camera, even some capacitor around was already soldered... And about i2c, looks like it's working fine: if I remove the camera VCC, "ceu_omni" shows me "camera not found"; putting VCC again, I've got "ov7670 found" (I edited the "I2C_BUS_SCAN_" routine)...

Chris is right, since the number of pins is limited, some of them are shared. So you need to manually tell your u-boot or whatever you want to physically connect VS, HS, CLK to your camera.
I2C is working (this is normal) but you're stuck here because there is no end of frame interrupt. You're not receiving this interrupt because you're missing at least one signal from your camera and CEU is still waiting for it.

Hi guys

Even changing "io_mux" (both pin "High"), no luck... I'll try to find a Scope so I can check the 3 pins signal...

Thanks again!

Hello guys

Now the camera is working! The problem was "RZA1_PINMUX" for CEU interface. Alternative function "f" is 6, according to datasheet, instead of 1:

...
<RZA1_PINMUX(10, 0, 6)>
...

Each capture is displaying on a simple webpage (slowly, but it's ok!).

@Chris, any chance to have this camera driver in kernel again?

The newer mainlined CEU driver is much better than the old one. But as Dylan pointed out, it didn't get in until 4.17.

We're in the process of backporting it to our 4.14 BSP. You can't just copy the driver back because some files in the video subsystem changed between 4.14 and 4.17. After we get it stable and tested, we'll push it to the 4.14 repo on github.

As for backporting it all the way to 4.9, we will probably not do that.

Thanks guys for the information!

So I'll wait the 4.14 update!

Thanks again!