Update: 2011-12-07 : SAM Reports 2 has been launched, as you may see on this blog post.

Accessing sliced log files

I’ve been working on a new version of SamReports for quite some time. My main goal was to provide seamless “integration” with Trixbox/Fonality SKU-s. (It’s not integration really, but the process of copying thousands of files from PBX, gluing them together, processing and so on, has to be transparent to the user, with as little hassle as possible, none preferably).

I came into possession  of a large dataset and that has helped  a lot. You cannot possibly simulate real, raw data to work with. You would just end up with gibberish. That brings the issue of processing..

Big data

Current version of SamReports tries to load all the data in memory, to generate reports. That just doesn’t work for millions of log lines.

The new version is capable of processing any amount of data, because it works in chunks, and generates reports sequentially. I tested on a dataset with more than 10 million lines in CDR, and it finished in 37 minutes.

Generating TrixBox and Fonality Reports

SamReports Parser for TrixBox and Fonality

Above is the image of a summary generated after processing the log files. The graphs are being drawn during the parsing so that you’re not bored while processing a large dataset for the first time :) This is displaying the same charts to get an overview.

Quirks and twists of master.csv (chopped or not)

I did get dirty with the local  and transferred calls in CDR logs. Sometimes a call can be spanned in 5, 6 log lines. Those lines don’t necessarily come close, but can be even 1000 lines apart.

master.csv calls on a local channel

local channels in master.csv

Here you see a call spanned into just 2 lines, one after the other.

To be continued…

This is just a preview of what’s been cooking. There are many other things to mention, but I will do it in another post…

Tags: , , , , , , , , , , , , , , , , , , , ,

4 Responses to “New version of SamReports will support Trixbox and Fonality and Big Datasets”

  1. Greg June 1, 2011 at 10:54 am #

    Hello, i am waiting for the Fonality version of product as we need a lot of reports about outgoing calls, their web interface do not provide the reporting for outgoing calls. What is your time frame to get the sam reports out ?

  2. admin June 1, 2011 at 11:03 pm #

    Thanks Greg. We are wrapping up the next release, and it supports Fonality as well as custom sliced log files structure. It will be available in a month, when we’re finished with all the testing and wrapping up. Thanks again for your interest.

  3. admin December 14, 2012 at 8:03 pm #

    I sent you an email.
    Thanks

Trackbacks/Pingbacks

  1. SAM Reports 2 is released! | Asterisk Reports Made Simple - SAM Reports - December 2, 2011

    [...] Reports 2 supports big datasets, as well as sliced (rotated) log files, as was introduced in this blog [...]

Leave a Reply

Clicky Web Analytics