There are many  changes between SAM Reports and SAM reports 2, as well as new features introduced.

The changes

• SAM Reports 2 supports big datasets, as well as sliced (rotated) log files, as was introduced in this blog post.
• The same goes to the viewer that displays the reports.
• Transfer of log files from the PBX/local PC now resumes, rather than copies all log files.
• Parser is now separated from the viewer (SAMReports.exe). That is to facilitate adding different Asterisk SKUs, that may be changing the log files in some way.
• Parser also shows a much more informative progress while processing.
• There is a different organization of Pivot Reports window:
  • SAM Reports 1 loaded all available reports for CDR and queues respectively, in the same window.
  • Now there is a command for each of the available reports containing different elements.

New features

• Pivot Reports
  • CDR Pivot Reports added an new field, “Destination Context“, as requested by some of our customers.
  • CDR and Queue Pivot Reports have a new charting engine, with three additional charts.
  • CDR and Queue Pivot Reports now support templates, that preserve current field layout. (Also requested by our customers)
• Detailed Reports
  • As requested by some of our customers, the detailed reports were introduced.
  • They display non-aggregated CDR and Queue calls.
  • Queue call drill-down reports were introduced:
    • Grid queue call drill-down
    • Flow chart queue call drill-down
• Advanced parsing
  • Sometimes an Asterisk log file( CDR mostly) contains a lot of redundant and unneeded lines. If those entered the reports they would only generate noise and clog the report.
  • We introduced a way to recognize and ignore such lines when generating report. There are also rules to assign agents an extension, if it is contained in their name.
• Log Files Viewer
  • Application that let’s you see the raw log files, as created by the Asterisk PBX.
• Parsing results
  • There is a new window that shows the last parsed results.
• Multiple Projects
  • SAM reports is not bound to a single Asterisk PBX anymore. It can generate reports for X number of Asterisk installations (X is determined by your SAM Reports 2 SKU : Enterprise-unlimited, Professional-10 and Standard-3) (Customer request)
  • You don’t need to make a new project only in case you have more than one Asterisk box. You might want to make a new project for each year, month…

I will be sending the update emails to my current customers next week. This upgrade is free for all registered users, and they will all be receiving a Professional SKU, an upgrade. I have decided to give free upgrades, because we consolidated all the efforts into this new version, and did not pay much attention to the version 1 of SAM Reports. That way we can thank them for being patient.

Related posts:

1. New Version is coming SOON! New features coming into SAMReports 2 : multiple projects, detailed...
2. New version of SamReports will support Trixbox and Fonality and Big Datasets The new version of SamReports is in the making, to...

Related posts brought to you by Yet Another Related Posts Plugin.

Project OverView

Now the home tab of SAMReports 2 displays the current project CDR and Queue aggregated chart data.

SAMReports Project Overview

Multiple projects

SAMReports Project Settings

SAMReports2 supports multiple projects. That means that you can report on multiple PBX-es, or/and add log files from the past. That would be the case if you had log files from (eg. 2007) some old Asterisk PBX, and the machine is not working anymore, but you would like to see what was happening at the time. Or you could make a new project for each year, or a month for the same Asterisk PBX.

Detailed Reports

SAMReports Asterisk Queue detailed Report

SAMReports now offers CDR and Queue detailed reports. Here you can see a sample Asterisk Queue detailed report.

Asterisk Queue Call Flow Chart

If you’re wondering what’s happening with your Queue Calls, you can dive into detailed reports and view a Queue Flow chart for a particular call.

Related posts:

1. SAM Reports 2 is released! The new version, SAM Reports 2 is out! There are...
2. New version of SamReports will support Trixbox and Fonality and Big Datasets The new version of SamReports is in the making, to...

Related posts brought to you by Yet Another Related Posts Plugin.

Accessing sliced log files

I’ve been working on a new version of SamReports for quite some time. My main goal was to provide seamless “integration” with Trixbox/Fonality SKU-s. (It’s not integration really, but the process of copying thousands of files from PBX, gluing them together, processing and so on, has to be transparent to the user, with as little hassle as possible, none preferably).

I came into possession  of a large dataset and that has helped  a lot. You cannot possibly simulate real, raw data to work with. You would just end up with gibberish. That brings the issue of processing..

Big data

Current version of SamReports tries to load all the data in memory, to generate reports. That just doesn’t work for millions of log lines.

The new version is capable of processing any amount of data, because it works in chunks, and generates reports sequentially. I tested on a dataset with more than 10 million lines in CDR, and it finished in 37 minutes.

SamReports Parser for TrixBox and Fonality

Above is the image of a summary generated after processing the log files. The graphs are being drawn during the parsing so that you’re not bored while processing a large dataset for the first time This is displaying the same charts to get an overview.

Quirks and twists of master.csv (chopped or not)

I did get dirty with the local  and transferred calls in CDR logs. Sometimes a call can be spanned in 5, 6 log lines. Those lines don’t necessarily come close, but can be even 1000 lines apart.

local channels in master.csv

Here you see a call spanned into just 2 lines, one after the other.

To be continued…

This is just a preview of what’s been cooking. There are many other things to mention, but I will do it in another post…

Related posts:

1. SAM Reports 2 is released! The new version, SAM Reports 2 is out! There are...
2. New Version is coming SOON! New features coming into SAMReports 2 : multiple projects, detailed...

Related posts brought to you by Yet Another Related Posts Plugin.

As I presented in my case, in the previous post, I need a virtual machine of an operational TrixBox PBX (Fonality edition).

• I need it to test connecting to the MySQL database in order to read the CDR and queue data.
• I also need it to further improve on my merging of the log files created by the system, and transferring them.

Privacy issues

• I’m aiming at PBXs that are no longer in use, but have not been refurbished yet.
• I will obfuscate the data. I made a little program, Randy, that generates random names and phone numbers, just for that purpose. I used it to change all the identifiable data from one of my clients. I’m using that data for demonstrating the features of Sam Reports, the owner of this blog.

What’s in it for you?

In Croatian (Ima’l mene tute?)

• If you’re interested in Sam Reports than you might benefit from a free license, once it’s out of beta and released.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

Update 2011-12-07 : SAM Reports 2 supports Trixbox/Fonality without any additional modules. Take a look at our new help files.

TrixBox (Fonality) issues

I would really like to make reading the reports from TrixBox (Fonality) a bit easier. Sam Reports generates Asterisk reports from log files that are present on every Asterisk box by default (master.csv and queue_log). TrixBox slices the log files into a bunch of small files and puts them in  date-named directories . Let me describe it in details:

Master.csv

It is originally just one cvs file located in /var/log/asterisk/cdr-csv/master.csv (I don’t care about the cdr-custom at the moment). On Fonality PBX, (I’m not sure that the situation is the same on all TrixBox SKUs), the master.csv data is placed in the same directory:  /var/log/asterisk/cdr-csv/09-10-01/,  but there are additional directories created there, on a weekly basis. So (the date – 09-10-01 – corresponds to YY-MM-DD). A new directory is created on Monday of each week , as stated in Fonality help files. I have copied all the files from Fonality PBX on my local PC, and below you can see a screenshot:

Fonality (TrixBox) master.csv

In each of these directories there are thousands of master.csv parts named: Master.csv.xxxxxxxxxx, where the xxxxxxxxxx is a UNIX time stamp (the number of seconds elapsed since Jan 1, 1970). One such file is created every 5 minutes, on a Fonality PBX. The file named just “master.csv“, that you see on the figure above, is the latest log that has not yet been stored in a directory and rotated.

Fonality-TrixBox master.csv parts

Queue_log

Originally queue_log file is located in /var/log/asterisk/queue_log.  On Fonality-TrixBox  it is also dissected  into many parts, in much the same way.

Fonality-TrixBox queue_log

The original “queue_log” file just holds the latest 5 minutes worth of information, and all other data is in files created by the same algorithm as CDR files.

Fonality-TrixBox queue_log parts

Merging the files

One of my beta testers wanted to try Sam Reports, but got only the 5 minutes worth of data. So I decided to make a little program to merge all the disjointed files into one, master.csv and queue_log respectively. If you ever have such need, you can download the file here: FonalityMerge.

• It requires that you have copied all the log files and directories from your Fonality box, somewhere on your local disk.
• Then you merge the files with FonalityMerge
• And you get your master.csv and queue_log

Now you can process the log files with Sam Reports and see your reports.

Further improvement

If I get enough inquiries for TrixBox/Fonality support I may opt to do one of the things:

• Make the transfer of directories from Fonality-TrixBox available with a click of a button. That would take into account just the diff between directories and files (the directories and files already present on the local disk would not be transferred any more)
• Or go for the database option and connect directly to the MySQL database with the logs. To do that I would need a working TrixBox with a database that contains meaningful data.  Meaningful means real data generated by using a system for 5-6 month, at least.

It would be great if someone could provide me with a virtual machine of a TrixBox PBX that has been operational for some time. I know it’s a long shot. I would be veeeery grateful.

Related posts:

1. New version of SamReports will support Trixbox and Fonality and Big Datasets The new version of SamReports is in the making, to...
2. SAM Reports 2 is released! The new version, SAM Reports 2 is out! There are...

Related posts brought to you by Yet Another Related Posts Plugin.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.