What’s the current state of the crowdfunding? Where are we, when it comes to our goals? What work has been done already? These are the question this article tries to answer.
TL;DR
- We did a lot of security updates
- Work on removing ActiveMQ is done
- Work on two of the three Solr indexes is done
- We have problems with the Spring update
ActiveMQ
The goal: Installing Opencast may be somewhat confusing to new users, partly because there are lots of different additional services to run. For a long time, one of them has been ActiveMQ which is a message broker used for inter-service communication in Opencast. Used… well… barely used, actually. With recent versions, we only needed ActiveMQ on a single server only. Since ActiveMQ is meant to distribute information across multiple servers, this meant we could also communicate with these services directly. In short, less overhead and fewer additional services to run for adopters. That is why our goal was to entirely remove Opencast’s dependency on ActiveMQ.
Current state: Work on this task has been mostly finished. A pull request removing ActiveMQ has been filed and reviewed. All that is left is a bit of cleanup work before it can be merged. This means that this is almost guaranteed to make it into the next major Opencast release.
Security Issues
github.com/opencast/opencast/security/advisories/GHSA-hcxx-mp6g-6gr9
github.com/opencast/opencast/security/advisories/GHSA-j4mm-7pj3-jf7v
github.com/opencast/opencast/security/advisories/GHSA-59g4-hpg3-3gcp
github.com/opencast/opencast/security/advisories/GHSA-mf4f-j588-5xm8
…
The goal: Opencast has a good track record of identifying and fixing security issues, and we had identified a few known or potential security issues we wanted to evaluate and fix, if they turned out to be problematic. That way we can keep our servers safe and avoid any spectacular data breaches.
Current state: There have been a number of security fixes for Opencast 9, 10 and 11. The issues we addressed range from limited data extraction, over privilege escalation to potential remote code execution attacks. Fixes for these have been included in the last couple of releases. We have also been able to dismiss a few reports of code we suspected to be problematic which turned out not to be a problem after all. Still, we have not yet processed the whole list of suspects. We will inform you, as usual, if we release another security patch and will keep trying to make these releases as responsible and painless as possible for adopters.
Log4j: We cannot talk about security fixes without pointing out one particular problem we faced as part of the crowdfunding. The Log4Shell remote code execution vulnerability and several additional vulnerabilities found in this library after the world’s security researchers all turned their attention towards Log4j have affected Opencast as well. We released several versions of Opencast in December to address these issues as fast as we could, since we knew that these vulnerabilities were actively exploited. To help adopters, we even decided to release new versions of Opencast 9 since it only just reached its end of life, and we knew about many adopters not having updated yet.
Solr
github.com/opencast/opencast/pull/3204
github.com/opencast/opencast/pull/3376
github.com/opencast/opencast/pull/3377
The goal: Opencast uses both Solr and Elasticsearch for full text search and caching. Both services serve an almost identical purpose. However, one of them is in desperate need of attention: Solr. We built an integration with Solr using an older version, which is both too old to easily deploy in a cluster, and not easy to update. In short, things have to change. But instead of updating Solr and still end up with two different services doing the same thing, we chose to consolidate on Elasticsearch¹.
Current state: Opencast uses Solr for three services: The series service, the workflow service and the search service. All of these services were user-facing in Opencast (Matterhorn) 1.x, which is why full text search and caching was important. The same is no longer true today, and thus the need for some of these indexes no longer exists.
We were able to completely remove two of the three Solr indexes, sparing adopters from re-indexing these ever again. The services this was done for are the workflow service and the series service. In the future, data will be requested from that database directly. The patches for these are currently being reviewed. We hope to get these merged soon to have them included in the next major new version of Opencast.
Work on the final service, the search service, is more complex and not yet done. We cannot remove Solr in the same manner, since full text search capabilities are actually used here and the service is still user-facing, being the back-end for the players among other things. We hope to still be able to make the shift to Elasticsearch for Opencast 12, but this is more challenging, and we will act with caution since it’s a central piece of Opencast infrastructure being used by all adopters.
[1] We may actually use OpenSearch instead of Elasticsearch, but that should be a drop-in replacement. We will report if it actually is. But for sake of simplicity, we stick to Elasticsearch for reporting.
Spring
The goal: Opencast uses Spring Security for handling logins and access control. We did fall behind when it comes to updating the library to its current state and are now using a version which is no longer supported. While this does still work just fine, unfortunately, like we have seen with Log4j, this bears the risk of suddenly blowing up. Thus we would like to update.
Current state: Our plan was to separate the different login mechanisms Opencast supports which are all woven into Spring Security, then start updating the core and basic login mechanisms first. At that point, we wanted to discuss further actions with the community.
Sadly, it turned out that this plan is not as easy as we hoped. Newer versions of Spring Security do not work well with our OSGi stack, and just updating even the core is not possible. Options we are now evaluating are investigating versions picked up by the Eclipse Gemini and Apache ServiceMix projects, which still provide supported versions but not the latest versions, and the possibility of support within Karaf itself which has been hinted at for the next major version but has not yet been confirmed.
Due to the not yet finalized statements about the Karaf roadmap, we decided to focus on the other tasks first, leaving this as the last potential task to tackle. The exact form of how we can/will tackle this problem and if we can completely fix this in this crowdfunding is still to be determined. We will make sure to start an open discussion about this once we have collected all information.
Questions; Next Steps
If you have any questions or want to discuss any of these tasks, don’t hesitate to bring this to the development mailing list, the Matrix chat or bring it up in the weekly technical meetings. Furthermore, if you want to help, consider reviewing any of the open pull requests linked above.
We will post again, once we have reached a new major milestone. Additionally, we will submit a session about the state of the crowdfunding at the upcoming conference. Join us there for a discussion, if you are interested.
Opencast 