We’re pleased to announce some major improvements to the OdataLink logs architecture.
What’s changed visually with the OdataLink log screen
First up, the biggest change we hope you’ll experience is that the logs page should be a lot more responsive than in the past. This is mostly due to the changes we made under the hood.
Secondly, the layout has changed slightly. We removed the ID column as it felt meaningless in the new system and we moved the Date column to the start.
Finally, we added the ability to filter the logs. This can be accessed via the Filters button. Clicking on it will allow you to select the date and the type of requests you want to see.
Logs can now only be viewed for one single day at a time (based on your timezone). You can change this date via the the Calendar filter. This calendar also highlights which days have logs and which do not.
In addition to the calendar filter, the Requests dropdown allows you to choose which type of requests to include.
All (including system calls)
This is the legacy option that shows all the calls taking place between your Odata Client (e.g. Excel, Power BI, Data flows, etc.) and OdataLink. It includes calls for the Service, Metadata and Data documents.
All (excluding system calls)
This new option is the default option. It only shows the actual request for data. It’s important to note that it will also include any errors that took place.
In Progress
This new option shows only the requests that are currently in progress so that you can track what is happening under the hood. It removes the noise of all requests that are complete.
Errors Only
This new option is similar to how the Home Page works and only shows errors.
What’s changed under the hood with the OdataLink logging system
The biggest change to the logging system, however, is something you cannot even see.
Under the hood, we’ve completely changed the technology the logs used.
Previously, logs were written to databases. While this made it easy to build with and query, it had a few flaws.
- The logging process was a bottleneck to performance.
Logging had to be processed somewhat linearly during a request which slowed the requests. - Logs from other accounts and users also impacted performance.
Logging shared many resources which created a secondary bottleneck. This not only impacted performance during logging but also when displaying logs. - Finally, it would be harder to support servers in multiple regions (hint of what is coming).
The new technology relies entirely on Azure Storage and addresses all three problems mentioned above.
What’s next in the OdataLink development pipeline
Over the next few weeks, we’ll monitor this new tech and address issues or improvements as they arise.
Behind the scenes though, we’ll get underway on the final milestone on the project we started many months ago. The improvements to the logging mechanism and the archiving mechanism were never the end goal. Those projects were merely stepping stone.
Stay tuned.