How to do LightGBM predictions in realtime at high scale in Java

We recently landed on a problem where we need to do predictions of LightGBM model in realtime in Java. Training in Python and Predicting in java is no new problem but this problem was unique because popular way of using PMML to do prediction was giving following issues.

Issues

• Validations at PMML generation end is very scrict even stricter then LightGBM. Some examples are
• PMML doesn't allow Single valued features while LightGBM do. 
• PMML have weird restrictions on value range which is strange. 
• At the time of prediction PMML predictor was doing some validation which was making the predictions very costly. 
• PMML generator is not able to parse feature values with special characters from model dump.
• PMML generation was taking 20 Min with 60GB of memory for 160MB model file. This is a big issue if we need to update the model every hour.
• Client of PMML has to be aware of data type of features.

We were able to find work arounds to some of above but not all and was in the bad state to scale it using PMML. Finally we gave up on PMML and

we wrote our own parser and predictor to parse model dump of LightGBM in Java and load self defined objects and do predictions. 

Problems faced : 

LightGBM uses its own format and representation of trees in dump of LightGBM. With no documentation along with fairly complex representation of Trees make this understanding hard. Understanding this dump was the primary thing refraining us from taking this approach from biggining. 

What we achived : 

• Memory requirements for loaded model is reduced by 50%
• Prediction time is reduced by 50%, cutting down server cost to compute prediction by 50%
• Model is more debugable, we can log anything we want, we are able to add break points and debug the model better.
• We are able to remove middle man PMML completely and along with dependency on external library.
• Significantly reducing resouce requirement and cutting time by 20 Min to complete the process.