Announcement: New Functionality in SPM and Logsene

Summer is all but officially over, yet our work with SPM Performance Monitoring, Alerting and Anomaly Detection and Logsene Log Management and Analytics is not.  While lots of us took a well-deserved break over the last 1-2 months, we added a few goodies to both SPM and Logsene.  More interesting stuff is coming in the next release.

New in SPM

With SPM, the most notable addition is monitoring for Apache Spark.  We’ll have a separate post about Spark monitoring with SPM next week with all the details, including screenshots.  But that’s not the only new goodness; other additions include:

Integration with Nagios

  • You can now tell SPM where your Nagios lives and SPM will push all your Alerts to Nagios.  If you use PagerDuty, SPM can push your Alerts there, too.

Lowered SPM agent overhead

  • Those sending large volumes of metrics will see the most benefit.  The new agent makes use of Apache Flume to transport metrics.

Switched to sending metrics over HTTPS by default

These additions to SPM, along with recently announced monitoring support for NGINX Plus and NGINX make it an even more effective solution for organizations who are paying the unfortunate price of having a mish-mash of monitoring and alerting tools bolted together in an uneasy coexistence.

If you haven’t seen SPM yet, we have a live SPM demo so you can see it for yourself.  The demo shows Hadoop, HBase, Kafka, Elasticsearch, Solr, MySQL, Redis, and other types of apps being monitored.

New in Logsene

Until now you could create an unlimited number of Dashboards with SPM graphs, and now you can do that with Logsene graphs, too.  Moreover, you can place Logsene log graphs alongside SPM’s performance graphs, on the same Dashboard, and correlate your performance with your application logs!

This makes the integration of performance metrics, logs, events and anomalies more robust for those of you looking to combine performance monitoring and centralized log management in one place — not only knowing that SOMETHING happened when you look at your performance metrics graphs, but also exactly WHAT happened by having immediate access to relevant logs right there!

Screenshot – Dashboard with SPM Performance Graphs & Logsene Log Graphs  [click to enlarge]

test_dashboard_SPM_Logsene

Take a Test Drive — It’s Easy and Free to Get Started

Like what you see here?  Sound like something that could benefit your organization?  Then try SPM or Logsene for Free for 30 days by registering here.  There’s no commitment and no credit card required.

Announcement: NGINX and NGINX Plus Monitoring in SPM

The list of monitored stacks keeps growing!  SPM Performance Monitoring, Alerting and Anomaly Detection just added support for NGINX and NGINX Plus.  Now you can capture metrics like total requests and connections along with an overview report and a whole lot more.

NGINX is a high performance, open source web application accelerator that helps millions of the world’s busiest websites deliver more content, faster, to its users.  It is the #1 web server for the top 10k busiest websites in the world. NGINX Plus is the commercial version which adds advanced features, professional services, and shows more juicy metrics.

NGINX and NGINX Plus monitoring in SPM complements recently announced support for apps like MySQL, Cassandra, Memcached, Apache and AWS CloudWatch.  You can see a more complete list here (scroll down to “Monitored Apps”)

Here’s a glimpse into what SPM for NGINX and NGINX Plus provides – click on an image to see the full view or you can look at the actual SPM live demo showing SPM monitoring NGINX and NGINX Plus and their metrics.

Screenshot – NGINX Plus Overview  (click to enlarge)

NGINX+ overview_new

 

Screenshot – NGINX Plus Server Status  (click to enlarge)

NGINX+ server status

 

Screenshot – NGINX Plus Status Zones  (click to enlarge)

NGINX+ status zones 2

 

Screenshot – NGINX Plus Caches  (click to enlarge)

NGINX+ caches

 

Screenshot – NGINX Plus Upstreams  (click to enlarge)

NGINX+ upstreams

 

Screenshot – NGINX Overview  (click to enlarge)

NGINX overview

 

Screenshot – NGINX Status  (click to enlarge)

NGINX status

 

Live Demo — See SPM for Yourself

Check out SPM’s live demo to see NGINX and NGINX Plus monitoring for yourself.  You’ll also be able to poke around and see Kafka, HBase, Elasticsearch, Solr, MySQL, and other types of apps being monitored.

Love the Idea of Monitoring NGINX and NGINX Plus? Take a Test Drive — It’s Easy to Get Started.

Try SPM Performance Monitoring for Free for 30 days by registering here.  There’s no commitment and no credit card required.

Logsene Log Management and Analytics Grows Up!

There are exciting changes on the immediate horizon for Sematext’s comprehensive logging solution: Logsene Log Management and Analytics.  We just announced the seamless integration of Logsene with SPM Performance Monitoring, Alerting and Anomaly Detection (more on that below) and we have a new release of Logsene just weeks away from production.  Here’s a glimpse of the new Logsene UI.  Watch for more log analytics and management goodies on our blog soon.

New Logsene UI sneak peek

logsene1366x768_blog_post

Of course, the new Logsene will keep the option to use the Kibana UI directly from Logsene.

New Logsene Functionality in Words

  1. We support multiple queries in a single view; each query’s data get added to the graph (hence 2 lines in this screenshot).
  2. Queries can be saved, converted to scheduled queries (get email with log graphs every day/week/month type of functionality), or to alert queries (i.e., “email me when we see log messages with XYZ in them”). Those little gray icons next to search fields are for that.
  3. Clicking on the icon to the left of the top search field will show more functionality.
  4. Filtering by various log attributes.
  5. Raw log events in what we internally call Logs Table (or LT). Multiple tabs on top correspond to multiple queries one can enter.
  6. On the left is a list of available fields that are shown in LT. Checking them adds them to LT. They can be dragged and reordered. Reordering them controls the order of columns in LT.
  7. Number behind field names shows the number of distinct values for that field.
  8. Logs can be downloaded as CSV or published to Github Gist or Pastebin services.
  9. Arrows in top-right of LT expand the LT into a “Kiosk mode.”
  10. Little icon in each row in LT can be clicked. It expands on click and lets you see context around the given log.
  11. Clicking on the two gray buttons in top right with down arrows will expose metrics from SPM and Events.

If you currently use Splunk or another log management tool then there is a good chance that you will find Logsene to be an easy and useful way to index, search and analyze your logs.

But Why Use Logsene?

In short: Logsene and SPM provide a single pane of glass for performance monitoring, centralized log management, alerts, anomalies, custom events, and custom KPIs.  Unlike competing products, SPM and Logsene together not only tell users that SOMETHING happened, it now tells them exactly WHAT happened.  No need for a mish-mash of individual open-source or commercial tools for monitoring, for alerting, for logging, for custom dashboards, etc., each with its own configuration, its own UI, etc.

Not only that, but Logsene offers a unique pricing plan that, unlike those of other log management products, does NOT charge for the amount of data and retention.  You only pay for the number of logs in the index – you don’t pay for data transfer or data retention.

We also offer Plan Auto-Upgrade, a unique feature that can automatically switch you to the next higher plan if you exceed your log threshold — preventing you from losing logs which could contain critical information, without forcing you to keep this higher plan.

Moreover, because Logsene usage is “by day”, when auto-upgrade happens the extra cost is really minimal — at any time after the auto-upgrade you can adjust retention and/or plan and go back to the old plan

Check out a Live Demo

See SPM and Logsene (note: old UI and Kibana UI, the new UI will be in the next release) for yourself by viewing a live demo.  You’ll also be able to poke around and see Storm, Kafka, Solr, Elasticsearch, Hadoop, HBase, MySQL, and other types of apps being monitored.

Try Logsene and/or SPM Performance Monitoring for Free for 30 days by registering here.  There’s no commitment and no credit card required.

We’re Hiring!

If you enjoy performance monitoring, log analytics, or search analytics, working with projects like Elasticsearch, Solr, HBase, Hadoop, Kafka, and Storm, then drop us a line.  We’re hiring planet-wide!  Front end and JavaScript Developers, Developer Evangelists, Full-stack Engineers, Mobile App Developers…get in touch!

Monitoring Kafka, Storm and Cassandra Together with SPM

Kafka, Storm and Cassandra — Big Data’s Three Amigos?  Not quite.  And while much less humorous than the movie, this often-used-together trio of tools work closely together to make in-stream processing as smooth, immediate and efficient as possible.  Needless to say, it makes a lot of sense to monitor them together.  And since you’re reading the Sematext blog you shouldn’t be surprised to hear that we offer an all-in-one solution that can be used for Kafka, Storm and Cassandra performance monitoring, alerts and anomaly detectionSPM Performance Monitoring.  Moreover, if you ship logs from any of these three systems to Logsene, you can correlate not only metrics from these three systems, but also their logs and really any other type of event!

Enough with all the words — here is an illustration of how these three tools typically work together:

Kafka-Storm-Cassandra

Of course, you could also be storing data into some other type of data store, like Elasticsearch or HBase or MySQL or Solr or Redis.  SPM monitors all of them, too.

So what do you get if you monitor Kafka, Storm, and Cassandra with SPM?

You get a single pane pane of glass, a single access point through which you can see visualizations of well over 100 metrics you can slice and dice by a number applications-specific dimensions.  For example, various Kafka performance metrics can be filtered by one or more topics, while some Storm performance metrics can be filtered by topology or worker.  Of course, you can set alerts on any of the metrics and you can even make use of SPM’s anomaly detection capabilities.

Kafka, Storm + Cassandra screenshot

 

Consolidate Your App Monitoring, Alerting, and Centralize Logging — It’s Easy!

Many organizations tackle performance monitoring with a mish-mash of different monitoring and alerting tools cobbled together in an uneasy coexistence that is often far from seamless.  SPM takes all that hassle away and makes it easy and comprehensive in one step.

Live Demo — See SPM for Yourself

Check out SPM’s live demo to see this monitoring for yourself.  You won’t find any demo apps showing Cassandra metrics because we don’t use it at Sematext yet, but you’ll be able to poke around and see Kafka, HBase, Elasticsearch, MySQL, and other types of apps being monitored.

Love the Idea of Monitoring Kafka, Storm & Cassandra Together?
Take a Test Drive — It’s Easy to Get Started.

Try SPM Performance Monitoring for Free for 30 days by registering here.  There’s no commitment and no credit card required.

We’re Hiring!

If you enjoy performance monitoring, log analytics, or search analytics, working with projects like Elasticsearch, Solr, HBase, Hadoop, Kafka, and Storm, then drop us a line.  We’re hiring planet-wide!  Front end and JavaScript Developers, Developer Evangelists, Full-stack Engineers, Mobile App Developers…get in touch!

 

Announcement: What’s New in SPM Performance Monitoring

A new SPM Performance Monitoring release was just pushed to production and it’s chock full of great new stuff to complement its proactive performance monitoring, alerting, anomaly detection, etc., available in the Cloud or On Premise.  Here is a run-down of the juicier additions. The slightly longer version can be found in the SPM Changelog.

Integration with Logsene Log Management and Analytics

SPM performance monitoring now gives users access to even more metrics by seamlessly integrating with event data and logs via Logsene Log Management and Analytics.  This enables correlation across performance metrics, alerts, anomaliesevents, logs, and provides a single pane of glass across any organization’s IT infrastructure.

Monitoring Support for More Applications

We’ve added native monitoring support for the following applications to complement monitoring for Solr, Elasticsearch, Hadoop, HBase, Storm, Redis, Kafka, ZooKeeper and many others.

Screenshots

Eager to see pictures instead of reading content?  Then jump below to see screenshots of these apps being monitored.

UI/UX Improvements

UI/UX improvements include: zooming and panning, client-side caching, wider and simpler metric charts, new filter slide-out panels with search capabilities, quick access to all dashboards and easier dashboard creation, and more.

Event Graphs

Events and event graphs are now integrated into SPM Performance Monitoring.  You can now correlate various types of events, such as alerts, anomalies, application deployments, restarts, releases, server reboots, etc., with performance metrics graphs, as well as with logs and log graphs.  Many of you will also be happy to hear that SPM can now turn Alerts into Events, and graph them as well.  Check out Event Integration if you want to publish your own Events.

More Powerful Storm Monitoring

SPM Storm monitoring now serves up more metrics, more graphs and provides more granular details.  This includes the addition of metric filters and the ability to monitor not just Spouts and Bolts, but also monitor Storm Workers.

Dashboard Enhancements

Creating and working with dashboards just got a lot more intuitive and flexible.  This includes:

  • creating new dashboards via an intuitive “build your own dashboard” tool
  • easier navigation via Miller Columns (think column-oriented view in OSX Finder)
  • adding whatever graphs you want to an existing or brand new dashboard from within that dashboard
  • a pull down menu to select specific dashboards for much quicker access to a specific dashboard

 

Screenshot – SPM Dashboard (one of many possible views; click to enlarge)

Dashboard_test

 

Screenshot – Cassandra Overview  (click to enlarge)

cassandra_overview

 

Screenshot – MySQL Overview  (click to enlarge)

MySQL Overview

Screenshot – Memcached Overview  (click to enlarge)

memcached-overview

Screenshot – Apache Monitoring Overview  (click to enlarge)

Apache Overview

 

Screenshot – AWS CloudWatch EBS Read/Write Bandwidth  (click to enlarge)

AWS_EBS Read:Write Bandwidth

 

Live Demo

Check out SPM’s live demo to see it for yourself.  You won’t find any demo apps showing Cassandra or Memcached metrics because we don’t use them at Sematext yet, but you’ll be able to poke around and see other types of apps being monitored — like Solr, Kafka, Hadoop and HBase, for example — along with MySQL, AWS, and Apache.

Consolidate Your App Monitoring — It’s Easy!

Many organizations tackle performance monitoring with a mish-mash of different monitoring and alerting tools cobbled together in an uneasy coexistence that is often far from seamless. SPM takes all that hassle away and makes it easy and comprehensive in one step.

Try SPM for Free for 30 Days

Try SPM Performance Monitoring for Free for 30 days by registering here.  There’s no commitment and no credit card required.

We’re Hiring!

If you enjoy performance monitoring, log analytics, or search analytics, working with projects like Elasticsearch, Solr, HBase, Hadoop, Kafka, and Storm, then drop us a line.  We’re hiring planet-wide!  Front end and JavaScript Developers, Developer Evangelists, Full-stack Engineers, Mobile App Developers…get in touch!

Announcement: Cassandra Performance Monitoring in SPM

Cassandra is a distributed database management system built to handle massive data sets while providing high availability without compromising performance.  That’s why many organizations use it for their mission-critical data. That being said, if you are running Cassandra then you’ll want to keep close tabs on it.  And now SPM Performance Monitoring can help you do just that as the latest release supports Cassandra performance monitoring.  Not to mention that all the usual SPM alerting, anomaly detection, etc., can be used with any of the Cassandra metrics.

Why Not Monitor More Than Just Cassandra?

Unlike some of the tools that only monitor Cassandra and nothing else, SPM Performance Monitoring covers Cassandra, the competing database HBase, and a lot more: Solr, Elasticsearch, Hadoop, MySQL, AWS CloudWatch, Memcached, Apache, and just about any other app you want to monitor.  SPM also monitors Storm and Kafka, which are often used together with Cassandra (considered their most popular data store).  Now all three can be monitored together!

Have a look at a few of the screenshots to see how we graph Cassandra metrics in SPM (a list of Cassandra metrics we monitor is listed further below).  You can also check out SPM’s live demo. You won’t find any demo apps showing Cassandra metrics because we don’t use Cassandra at Sematext yet, but you’ll be able to poke around and see other types of apps being monitored, like Solr, Kafka, Hadoop and HBase, for example.

Overview  (click to enlarge)

cassandra_overview

 

Compactions  (click to enlarge)

compactions

 

Pending Writes  (click to enlarge)

pending-writes

 

Write Requests  (click to enlarge)

write-requests

 

SSTable  (click to enlarge)

sstable

 

Bloom Filter  (click to enlarge)

bloom-filter

SPM now monitors Cassandra metrics like:

  • Write Requests (rate, count, latency)
  • Read Requests (rate, count, latency)
  • Pending Write Operations (flushes, post flushes, write requests, replication of write)
  • Pending Read Operations (read requests, read repair tasks, compactions)
  • Pending Cluster Operations (manual repair tasks, gossip tasks, hinterd handoff, internal responses, migrations, misc tasks, request responses)
  • Compactions
  • Row/Key Cache (hit ratio, requests count)
  • Local Writes (rate, count, latency)
  • Local Reads (rate, count, latency)
  • SSTable (size, count)
  • Bloom Filter (space used, false positives ratio)

Please tell us what you think – @sematext is always listening!  Is there something SPM Performance Monitoring doesn’t monitor that you would really like to monitor?

Consolidate Your App Monitoring — It’s Easy!

Many organizations tackle performance monitoring with a mish-mash of different monitoring and alerting tools cobbled together in an uneasy coexistence that is often far from seamless.  Think Graphite+Nagios, for example.  SPM takes all that hassle away and makes it easy and comprehensive in one step.

Try SPM for Free for 30 Days

Try SPM Performance Monitoring for Free for 30 days by registering here.  There’s no commitment and no credit card required.

We’re Hiring

If you enjoy performance monitoring, log analytics, or search analytics, working with projects like Elasticsearch, Solr, HBase, Hadoop, Kafka, and Storm, then drop us a line.  We’re hiring planet-wide!  Front end and JavaScript Developers, Developer Evangelists, Full-stack Engineers, Mobile App Developers…get in touch!

Announcement: Apache Monitoring in SPM

Responsible for your organization’s Apache servers?  If so, you’ll be happy to hear that you can now capture metrics about your Apache web servers — along with alerting and anomaly detection, traffic spikes, dips/gaps, etc. — with SPM Performance Monitoring.  This includes system-wide visibility into resource utilization, application performance, and operational health.

We’re also announcing other great new additions to SPM in the coming days to complement just-released monitoring support for MySQL, Cassandra, Memcached and AWS CloudWatch.  Watch this space for details…

Here’s a glimpse into what SPM for Apache provides – click on an image to see the full view or you can look at the actual SPM live demo showing some of our own Apache servers and their metrics.

Overview  (click to enlarge)

Apache Overview

 

Workers  (click to enlarge)

Apache Workers

 

Traffic Rate  (click to enlarge)

Apache Traffic Rate

 

Scoreboard Serving  (click to enlarge)

Apache Scoreboard Serving

 

CPU Details  (click to enlarge)

Apache CPU Details

 

Memory Details  (click to enlarge)

Apache Memory Details

 

IO Read/Write  (click to enlarge)

Apache IO Read:Write

 

Network Traffic  (click to enlarge)

Apache Network Traffic

 

Please tell us what you think – @sematext is always listening!  Is there something SPM Performance Monitoring doesn’t monitor that you would really like to monitor?

Why Have Something to Just Monitor Apache?

…When you can monitor almost everything with one solution — SPM.  Many organizations tackle performance monitoring with a mish-mash of different monitoring and alerting tools cobbled together in an uneasy coexistence that is often far from seamless.  Think Graphite and Nagios just for Apache, and numerous other tools for…everything else.  SPM takes all that hassle away and makes it easy and comprehensive in one step.

Try SPM for Free for 30 Days

Try SPM Performance Monitoring for Free for 30 days by registering here.  There’s no commitment and no credit card required.

We’re Hiring!

If you enjoy performance monitoring, log analytics, or search analytics, working with projects like Elasticsearch, Solr, HBase, Hadoop, Kafka, and Storm, then drop us a line.  We’re hiring planet-wide!  Front end and JavaScript Developers, Developer Evangelists, Full-stack Engineers, Mobile App Developers…get in touch!

Follow

Get every new post delivered to your Inbox.

Join 1,668 other followers