WLAN and SPI: going native

[andrzej zaborowski]

2008/9/30 Werner Almesberger <werner at openmoko.org>:
> It's been a while since my last update. I have some good news and some
> bad news. First, let me show an overview of the stack that handles the
> communication between the AR6k driver and the AR6k SDIO module. Each
> line represents a possible (compile-time) configuration of the stack:
>
> AR6k(Atheros) -> HIF(Atheros) -> SDIO(Atheros) -> S3C SDIO(Samuel)
>              -> HIF(Werner) -> SDIO(Linux) -> MMC-SPI(Linux) -> S3C SPI GPIO
>                                                              -> S3C SPI (poll)
>                                                              -> S3C SPI (int)
>                                                              -> S3C SPI (dma)
>                                            -> S3C MMC/SD/SDIO
>              -> HIF(?) -> G_SPI ...
>
> What the Openmoko kernel is using at the moment is the configuration
> in the first line: The AR6k driver from Atheros connected to the
> Atheros SDIO stack, and then the S3C SDIO driver Samuel Ortiz wrote
> for Openmoko.
>
> We're trying to find out if we could also attach WLAN via SPI instead
> of SDIO, and - more importantly - how we can use the Linux mainline
> SDIO stack instead of Atheros' stack (which would be unsuitable for
> mainline inclusion).
>
> A while ago, I succeeded in replacing the HIF interface between the
> AR6k driver and SDIO with a version that connects to the Linux SDIO
> stack instead of the Atheros stack, using SD(IO) SPI mode, and Ben
> Dooks' SPI GPIO driver to talk to the WLAN chip [1].
>
>
> The next step, using the SPI hardware in the S3C (using Ben Dooks'
> driver), turned out to be much harder than expected. I found the
> following three problems:
>
> - when enabling the SPI hardware, it generates a stray interrupt that
>  confused the driver's internal synchronization and made it overrun
>  the SPI hardware. This is already fixed in upstream, but I missed
>  the change when comparing the diffs :-(
>
> - SPI always sends and receives one data unit (e.g., a byte) at the
>  same time. The S3C's SPI hardware seems to have a flaw in its receive
>  buffer than makes it deliver the received byte only after the next
>  byte has been transmitted.
>
>  I'm not sure if this happens only in the 2442 or if other S3C are
>  affected as well.
>
>  The S3C SPI driver assumes that the nth byte received is available
>  after the nth byte has been transmitted, so it shifts the receive
>  buffer by one byte. This is okay for many SDIO commands, but it
>  causes the kernel to miss the response token after commands which
>  are followed by a data block.
>
>  A tentative fix is in [2].
>
> - This allows me to use the native SPI driver for SDIO in SPI mode up
>  an SPI clock rate of 12MHz. At speeds of 13MHz or above, the S3C
>  SPI hardware seems to sample the MISO signal a bit too early, and
>  byte sequences sent by the WLAN module thus appear to be shifted by
>  one bit to the left.
>
>  I'm not entirely sure if this is just an effect of my hardware setup
>  or if it's a real bug in the 2442. However, on the scope, the timing
>  of MISO seems impeccable. Changing SPI mode from 0 to 3 and removing
>  the probes (capacitative load) did not change the results, i.e., it
>  works at 12MHz but not at 13MHz.
>
>  There's another mystery, and that's that I think I actually had SPI
>  run at 25MHz without ill effects, but that was in interrupt mode,
>  not poll mode (see below).
>
> Using the S3C's SPI hardware to communicate with WLAN requires a
> connection from the SPI0 port (which is available on test points near
> the debug connector) to the connector of the WLAN module, which can
> be accessed on test points under the LCM. This connection can be easily
> made with a ribbon cable, as shown in [3]. I also wrote a script that
> tests the connections [4], using the gpio utility [5].
>
> When making this rework, the GPIOs also need to be reconfigured such
> that SDIO and SPI don't end up driving the now shared lines. This is
> accomplished by the patch in [6]. (If anyone wants to reproduce the
> setup, there are some more patches you need. Please look at the whole
> set in http://svn.openmoko.org/developers/werner/wlan-spi/patches/)
>
> This was with the S3C SPI driver operating in interrupt mode. I
> obtained a throughput of 100-110kB/s using SCP over a WEP-encrypted
> link to my access point. This is much better than the ~20kB/s I got
> with bit-banging S3C SPI GPIO, but still well below the ~1.2MB/s we
> should be able to obtain over SDIO SPI at 12MHz.
>
> A look at what's happening on the wire [7] (from the bottom up, we see
> clock, MOSI, MISO, interrupt, SS, and the unused DATA2) makes it clear
> why this is so slow: after each byte, the S3C SPI hardware generates an
> interrupt, and transmission stops until the CPU handles that interrupt
> and puts the next byte into the transmit register. This causes enormous
> delays between bytes.
>
> Rewriting the driver to use poll mode was easy [8], and the result
> looks much friendlier [9]. Since we're now approaching speeds where the
> CPU overhead of SCP makes itself felt, I switched to good old ttcp as
> the testing tool. In interrupt mode, I now obtained about 190 MB/s

I'm affraid that was kB.

> transferring data (that was immediately discarded). Poll mode delivered
> a much more satisfying ~385kB/s. (*)
>
> Can we go faster ? But of course. First, the driver could use DMA mode,
> which gives the CPU time to do more useful things than busy-waiting
> until the SPI hardware is done shifting bits. Since DMA mode has a
> relatively high setup cost and also includes an interrupt, I wouldn't
> expect it to yield a dramatic speed increase, but it should still make
> things work a little better.
>
> Second, the Linux SDIO stack uses a phantastically inefficient method
> for detecting interrupts from the SDIO device: a kernel thread
> periodically issues an SDIO command to read the device's interrupt
> register and then calls the interrupt handlers according to the result.
>
> This may make sense with SDIO controllers that don't properly support
> the somewhat twisted SDIO protocol for signaling an interrupt, but in
> SPI mode, we have a dedicated interrupt line for just this purpose ...
>
> What's left ? SDIO in SPI mode is of no direct use for GTA02, so I'll
> now look into making this work with the S3C MMC driver. I had already
> tried the S3C MMC driver at the very beginning, but it didn't work.

I seem to remember Samuel got it half working at some point.  IIRC at
some moment he said he had made the wlan initialise with the native
stack and only some hours later he actually registered to an access
point.  IIRC that work hadn't been finished and then there were higher
priority things to do when the newer firmware version came.  It might
be worth searching his in his directories on http://*.openmok.org/

Cheers