Project Bolt: Search-Driven API

It is time to introduce our second project at Ucommerce, Project Bolt. A project that has been in demand for quite a while now, which is why it is awesome to finally reveal that we are working on it. Keywords are; unification, performance, safe by default, and search.

So why Bolt? Like Project Kondo, our developers have been inspired by a person who is outstanding within his field. This time, the former Jamaican sprinter, Usain Bolt, has been the inspiration for our new API due to his record-breaking speed. The purpose of Project Bolt is to develop a new API that the developers use to show product data on the frontend where performance (a.k.a. speed) is key! Another characteristic of the new API is that it contains everything you need as a developer in one unified API, making search and querying much more straightforward compared to our current APIs. 

At our Partner Summit a couple a weeks ago, Morten Skjoldager, Senior Software Developer and part of team Bolt, presented the project to the audience - find the video below (remember to apply the subtitles. Again, we apologize for the poor audio quality). 

Today’s APIs

Over the years, we have come to the realization of some areas of our current APIs which have caused trouble in some way or the other. The issues we have discovered have then served as a foundation for what must be improved, fixed, or removed together with vast feedback collected from our partners and end-users. Our current APIs have served us well and been widely used and accepted, however, they have not received enough tender loving care from us. A primary issue has been that the APIs for the search were not fully baked into the platform, making a search-driven architecture hard to achieve as they were not compatible with the APIs for doing everything else. This has naturally caused confusion and pitfalls. The same level of confusion has been caused by the existing APIs as something could be done in multiple ways e.g. loading a product. Which one should you use?  

Another flaw of our current APIs relates to the amount of work needed to achieve satisfactory performance. That will change we are making powerful performance an out-of-the-box feature going forward. 

 

Unified and Super Fast: The New API

So, how have we gone about addressing these issues? We have pinpointed four focus areas of the new API. 

  1. Performance Of course, performance is a number one priority, which we are assuring by measuring both index time and query time compared to the current APIs. 
  2. Open & Extensible The true DNA of Ucommerce that we will never abandon, just improved radically. Plugging in a new search technology should be achievable without too much work involved.
  3. Code UX Is that even a thing? Well yes. The feedback we have collected reveals that APIs become cluttered very easily. Code UX refers to the amount of effort put into making sure it is seamless and easy to work with.
  4. Stellar Support Fear not! You will be in good hands. Things that break will have a clear migration path making sure you can get started with the new stuff as soon as possible.

For the new API, we do not use LINQ. Instead, we have chosen the best bits to make sure that we can actually live up to what we ship. A full blown LINQ provider for a set amount of new search databases is quite a mouthful and you probably don’t need all of it. This is why we’re introducing a light version of LINQ that looks and feels natively to what LINQ is just more simple, yet still very powerful. 

 

A Word on Search

We mentioned that the new API is search-driven, but what does that entail? 

At its core, it’s a way of expressing the capabilities of the new API and embracing the fact that search is default in any commerce solution today. So naturally, conducting a search and finding the correct data should be as easy as possible. This is also why we have a strong focus on delivering the different search capabilities like faceted search, full-text search, fuzzy search, weighted search, and more into the default set of features in the new API. In other words: search should be a default, not something you plugin on the side, where it is not native to the core API. The big news is that we are saying goodbye and sending our best of luck to RavenDB, which will not be included as out-of-the-box of Ucommerce anymore. Instead, we are welcoming Lucene and Elastic Search to UcommerceLucene will be shipped as an out-of-the-box search engine. It can be rolled out together with the installation without developers doing anything else. Also, Lucene is a well-tested and popular tool for read-fast capabilities. In fact, you can take Lucene very far, thus we are aiming to support most use-cases with Lucene alone. However, as you might know, Lucene is not suited to scale with the business to comprehend enormous and complex webshops with multiple servers. For this, Elastics is the go-to with its awesome capabilities to scale out. With the two search engines combined, we believe that we can bring the API all the way to a first-class system. 

 

Making Sure We Hit the Pit of Success

All things mentioned so far sound really good (at least we think that), but to ensure success, we have established four design goals to guide us in the process. 

  1. No Entities Surfaced in the API - Going forward, entities will be removed and replaced with identifiers to make a more flexible but (initially) less convenient API while transitioning to the new stuff. Having NHibernate entities surfaced in the API along with an API that has no entities at all, is tedious and annoying and only brings confusion to the table. So all new APIs will be replaced with either ids, guids, or something else. 
  2. Context Independent - Our context APIs has served us well over the years. Having them automatically resolve what a customer is looking at right now is awesome. However, as more and more commerce solutions already have that context in the client (browser), and just want to use our APIs in REST or RPC-styled web services where context is already known, it is not needed. In fact, it can be in the way quite a lot. So they will both be context-independent and Http context-independent. Though, in case you need it it will be there. 
  3. Scalable Through Pluggability - When your shop outgrows Lucene, you can just enable Elastic Search instead. The new API will work seamlessly across the two technologies. As always, disabling or enabling the two choices of technology will be a matter of switching configuration. 
  4. LOTS of Breaking Changes - We have awesome things lined up in the pipeline that will make Ucommerce so much better. So without revealing too much (we are still designing, and it is still a little too early to say), there will be lots of breaking changes. Maybe not in Project Bolt v1 but down the line, we will remove the old APIs to ensure you do not run into the old traps and limit the confusion.

 

So, when can you expect all of this? With everything else in software, it is difficult to say because things are subject to change. However, we are expecting to release version 1 within the first quarter. Version 1 is set to include the “safe by default” API, indexing, data-driven architecture choices, catalog scenarios support, and Lucene as the search provider. Though, we can promise one thing  it is going to be awesome!


Show more
{{lineitem.ProductName}}

{{lineitem.VariantName}} - {{lineitem.Quantity}} x {{lineitem.FormattedPrice}} {{lineitem.FormattedPrice}}

Your cart is empty ;(
Total {{basket.FormattedProductsTotal}}
Checkout