Qualcomm's very first open ISP driver
When we first approached their IoT camera team six years ago, Qualcomm was seen as an engineering house managed by lawyers. It was so clear that opening their camera stack was very far from their roadmap that I believed Qualcomm would be the very last company to adopt libcamera (likely in a tie with Nvidia). We nonetheless stayed in contact, trying to hammer down the same openness message year after year.
Stop-Gap Measures
In the meantime, Ideas on Board worked with Linaro and Red Hat to develop a software ISP implementation in libcamera. If we couldn’t leverage the hardware ISP in Qualcomm’s SoCs, we wanted to provide open-source users with at least the ability to capture raw images and process them in software. Qualcomm laptop owners and the PostMarket OS community received a stop-gap solution for video conferencing, at the cost of increased CPU usage and power consumption.
The effort continued with GPU acceleration of the software ISP, to improve performance and implement features too costly for the CPU. The work is still ongoing, with promising results, but requires a powerful-enough GPU that not all IoT devices provide.
A Shift
Two years ago, Qualcomm announced a strategic shift towards growth in IoT. Unlike the traditional smartphone market where SoC vendors can afford to directly support a small number of high-volume customers, IoT brings millions of low-volume customers. This requires a completely different approach to technical support, designed around open-source and upstreaming. Showing both understanding of the issue and commitment to solve it, Qualcomm started hiring a massive number of engineers with upstream expertise to implement an open-by-default strategy. That signalled the beginning of what everybody knew would be a long journey.
With the narrative that we had been pitching for multiple years now being adopted internally, cautious hope for Qualcomm support in libcamera was permitted once again and we doubled down on our efforts.
Enter Arduino
When Qualcomm announced their acquisition of Arduino, they pledged to preserve the products’ open approach. The community reacted with a mix of excitement and concern, especially after the update to the Arduino Terms of Service. To Qualcomm’s defense, commitment to openness can’t be proven overnight.
In a move that shows their strategic shift towards upstreaming is real, Loic Poulain from Qualcomm has just posted a patch series titled [RFC PATCH 0/3] media: qcom: camss: CAMSS Offline Processing Engine support to the linux-media mailing list. This is a giant leap forward whose importance can’t be overstated: not only does it pave the way for camera support in the Arduino Qualcomm ecosystem, it also opens the door to using the hardware ISP in a broader range of platforms.
Ideas on Board will keep working with Qualcomm to integrate the Qualcomm ISP in libcamera, cover a wider range of SoCs, and extend the supported features. The road ahead is still long, but the future looks brighter.