Performance Monitoring Comparison: Build vs. Buy

Using a performance monitoring system that you built yourself? You are not alone!  Many organizations monitor their applications and IT infrastructure with a bolted-together and often incompatible assortment of tools.  With larger organizations this can number to a dozen or more different tools.  Seriously.  Build vs. Buy, Do-It-Yourself (DIY), homegrown, in-house, Not Invented Here (NIH) — there are almost as many terms to describe this approach as there are products to do the monitoring.

There’s a good chance you’re using tools like Statsd, Graphite, Nagios and others to stay on top of things.  But that’s a LOT of work.  And why spend all the time doing all that work yourself?  Life, as we all know, is too short.  Is glueing together N tools or building yet another custom monitoring tool really a good use of (y)our (life)time?  This also leads to the next obvious question:

Why Not Use One Monitoring Solution to Do It All?

SPM Performance Monitoring, Alerting and Anomaly Detection is a comprehensive solution that does the work of many individual monitoring tools in one powerful package.  Applications, servers, other key IT devices — even logs! — are all covered.  A partial list of monitored apps includes Elasticsearch, Solr, Hadoop, HBase, Spark, Cassandra, Kafka, Storm, Redis, NGINX Plus and NGINX.  You can even see what I’m talking about right now by checking out our SPM live demo.

In fact, as one SPM user recently told us:

“I don’t want to be a data ape and consume your data to build other reports.  I think that is one of the attractions with SPM — I can push the data to Graphite or another monitoring tool, but you already have the reports done. So my time to insight is much faster.”

There Are Some Huge Differences Between Building and Buying

If your Building approach is draining engineer time that could be better spent elsewhere, then you should consider some of the key differences between building your own monitoring “system” and using SPM, including:

  • Log & Event Correlation: SPM can aggregate, graph and correlate logs with performance metrics and alerts (via integration with Logsene Log Management and Analytics).  If you are managing your logs then you are using a separate solution that does not integrate with your “Build” monitoring system.  Being able to see logs along with performance metrics is essential for effective troubleshooting.
  • On Premises or in Cloud: SPM offers an On Premises version in addition to SaaS.  Most app-specific monitoring tools are SaaS-only, but some organizations like their metrics and logs close to home base.
  • Native App Monitoring vs. 3rd Party Plugins: SPM monitors all apps natively.  If you are monitoring a number of individual apps via a range of 3rd party plugins then you have to deal with multiple installation and data collection mechanisms, various levels of maturity, and widely varying qualities of implementation and of reporting.
  • Anomaly Detection: SPM has support not only for heartbeat alerts and threshold-based alerts, but also for automatic machine learning based anomaly detection.  A Build system most likely does not have comprehensive anomaly detection capabilities.

And Then There is the Cost of Using All Those Different Monitoring Tools…

Cost comparisons between Building your own monitoring system and Buying a solution like SPM are not linear.  While some monitoring tools are open-source and free (though the time to configure them can be costly in its own right), commercial tools run a wide gamut of costs, infrastructure limits, data limits, time limits, pricing schemes, etc.  Just keeping track of the costs is often a job in itself.  In general, the more tools you have, the more value SPM delivers.

Here’s one scenario that will give you an idea of potential Build costs:

Build Your Own Monitoring System — Cost Scenario

  • Hourly rate:        $100 (ballpark figure; could be much higher)
  • Installation:        2 hours (very optimistic)
  • Configuration:   8 hours (very optimistic)
  • Maintenance:    2 hours/month (optimistic)
  • Upgrading:        2 days (i.e., ~20 hours)/year (IF all goes well!)
  • # of servers to run this configuration:  3 (monitoring 10 total servers*)
  • Cost per server (hardware): $1,000 each (i.e., $3,000 total)

___________________________________________________________

  • Total Cost in Year 1:        $6,200
  • Total Cost in Year 2:        $3,200 (not including any additional server purchases)
  • Total Cost in Year 3:        $3,200 (at least, though most likely higher)

And we didn’t even count the time cost to actually learn how to use all these tools!

Moreover, we used very optimistic numbers, assumed nothing will go wrong, assumed no issues like backwards incompatibilities, problems around dependencies, etc. etc. – all issues that are actually very common and can consume days and make the above costs much higher.  We do a ton of DevOps work at Sematext and, like everyone in this field, know how common this is.

* this number can vary widely, but for example purposes, if you want a complete monitoring solution that can do everything SPM can do — monitoring, alerting, anomaly detection, graph emailing, embedding, etc. — then the total servers that can be monitored with 3 monitoring servers will be lower than it would with a bare bones, incomplete monitoring tool.

SPM — Cost Scenario

  • # of servers: 10 servers (for example purposes)
  • Standard plan (our lowest cost plan beyond Free): $25/server/month
  • Time to Register and Install N agents: 1 hour (or $100 at hourly rate)

________________________________________________________________

  • Total Cost in Year 1:        $3,100
  • Total Cost in Year 2:        $3,000
  • Total Cost in Year 3:        $3,000

And these costs don’t even include any Volume Discounts that we would offer!

You can find clear, simple pricing plans for SPM right here.

Conclusion

While it’s great that there are many tools available for monitoring — some of them free — and communities built around those tools, in the DevOps world it still comes back to time.  Time to learn these tools.  Time to stay up-to-date on them.  Time to deploy.  Time to configure.  Time to maintain.  Time to assemble a bunch of disparate tools so you can monitor more than just one app.  You get the picture.  And time, as we all know, carries a cost.  With DevOps this is typically a significant cost.  So before undertaking a long and neverending “Build” journey, it makes sense to look at all the costs — in money and time — of buying a complete monitoring solution like SPM vs. building your own system. The closer you look, the more value a tool like SPM offers you and your organization.

Try SPM for Free for 30 Days

Tired of building, and building, and building…  Try SPM Performance Monitoring for Free for 30 days by registering here.  There’s no commitment and no credit card required.

Apache Spark Monitoring in SPM

Apache Spark is an open-source, large-scale data processing engine built on top of the Hadoop Distributed File System (HDFS) and enables applications in Hadoop clusters to run up to 100x faster in memory, and 10x faster even when running on disk.  So it’s not surprising the usage of Spark is booming as this Google Trends graph shows.

And while Spark usage has been going through the roof, Engineers and DevOps handling Spark have not had a good monitoring tool at their disposal.  Well, that is, until now.  By releasing the first Spark monitoring product to market Sematext has, with the addition of Spark monitoring to SPM Performance Monitoring, Alerting and Anomaly Detection, just filled a big hole in the Spark ecosystem.

Having just been added — along with other goodies — to the latest SPM release, SPM for Spark monitors all Spark metrics.  It includes alerting, anomaly detection, log correlation, custom dashboards, events graphing, custom metrics, and a ton more.  SPM can be installed On Premises or one can use the Cloud version run by Sematext, in which case the setup takes less than 5 minutes before graphs with performance metrics start appearing in real-time.

Enough with the words – Show me what Spark Monitoring looks like!

Have a look at a few screenshots to see how we graph Spark metrics in SPM.  While we don’t use Spark at Sematext at this time and thus don’t have a live demo to show you, you can check out SPM’s live demo and see some other types of apps we monitor, such as Hadoop, HBase, Cassandra, Kafka, Storm, ZooKeeper, Elasticsearch, Solr, NGINX and NGINX Plus, Apache, MySQL, Redis, Java webapps and generic Java applications, as well as custom metrics.

Screenshot – Spark Executor metrics [click to enlarge]

Spark_screenshot_Executor_3

 

Screenshot – Spark Worker metrics  [click to enlarge]

Spark_screenshot_Worker_2

And One More Thing…

SPM now works hand-in-hand with Logsene Log Management and Analytics.  This makes the integration of performance metrics, logs, events and anomalies more robust for those of you looking to combine performance monitoring and centralized log management in one place — not only knowing that SOMETHING affected performance of your Spark cluster when you look at your performance metrics graphs or get an alert, but also exactly WHAT happened with the cluster by having immediate access to all relevant Spark event logs right there!

Take a Test Drive — It’s Easy and Free to Get Started

Like what you see here?  Sound like something that could benefit your organization?  Then try SPM and/or Logsene for Free for 30 days by registering here.  There’s no commitment and no credit card required.

Correlating Metrics and Logs — Use Case: Elasticsearch Indexing

Here’s one way users can benefit from the SPM Performance Monitoring, Alerting and Anomaly Detection and Logsene Log Management and Analytics integration we just announced in the latest release.

Problem – CPU Utilization hits 95%!

  • You get an alarm about a CPU usage jump to 95% (note: using classic threshold-based alerts for CPU usage is a little crazy.  SPM’s anomaly detection feature would be a much better thing to use for CPU usage metrics).
  • You wonder, naturally, why this is happening and investigate immediately.
  • Without access to log graphs — like you would have with an SPM and Logsene combination — you would not be able to tell right away that the indexing rate increased.  It could be anything.  So you would need to connect, via ssh or VPN, to a server (or servers) where the CPU jumped and start looking around and see which process has been using the most CPU.  You’d run tools like top, vmstat, etc., but of course they’d have no historical data.
  • Even knowing which process uses the most CPU is not detailed enough.  You need to start looking at logs — either in another vendor’s log management tool which does not work seamlessly with your monitoring tool or manually “grepping” through one or more potentially very large log files on one or more servers — and try to determine what this application is doing more of now than it did before.  Not surprisingly, this is error-prone, time-consuming, and needlessly manual.  Most people have better things to do and want better tools.

Solution: Use SPM and Logsene Together to Triage

With a dashboard like the one you see here you can quickly tell what happened — i.e., why CPU usage went up.   In this particular case it is because the Elasticsearch indexing rate increased.  Now that the problem has been identified you can move on to taking action to fix it if a fix is needed.  Note:  You can even access the actual logs via Logsene so you can really be sure that there is no increase in some errors that are related to higher CPU usage.

test_dashboard_SPM_Logsene

We hope you found this use case helpful.  Got other performance monitoring, centralized log management or search-related use case ideas you’d like to see?  Drop us a line!

Announcement: New Functionality in SPM and Logsene

Summer is all but officially over, yet our work with SPM Performance Monitoring, Alerting and Anomaly Detection and Logsene Log Management and Analytics is not.  While lots of us took a well-deserved break over the last 1-2 months, we added a few goodies to both SPM and Logsene.  More interesting stuff is coming in the next release.

New in SPM

With SPM, the most notable addition is monitoring for Apache Spark.  We’ll have a separate post about Spark monitoring with SPM next week with all the details, including screenshots.  But that’s not the only new goodness; other additions include:

Integration with Nagios

  • You can now tell SPM where your Nagios lives and SPM will push all your Alerts to Nagios.  If you use PagerDuty, SPM can push your Alerts there, too.

Lowered SPM agent overhead

  • Those sending large volumes of metrics will see the most benefit.  The new agent makes use of Apache Flume to transport metrics.

Switched to sending metrics over HTTPS by default

These additions to SPM, along with recently announced monitoring support for NGINX Plus and NGINX make it an even more effective solution for organizations who are paying the unfortunate price of having a mish-mash of monitoring and alerting tools bolted together in an uneasy coexistence.

If you haven’t seen SPM yet, we have a live SPM demo so you can see it for yourself.  The demo shows Hadoop, HBase, Kafka, Elasticsearch, Solr, MySQL, Redis, and other types of apps being monitored.

New in Logsene

Until now you could create an unlimited number of Dashboards with SPM graphs, and now you can do that with Logsene graphs, too.  Moreover, you can place Logsene log graphs alongside SPM’s performance graphs, on the same Dashboard, and correlate your performance with your application logs!

This makes the integration of performance metrics, logs, events and anomalies more robust for those of you looking to combine performance monitoring and centralized log management in one place — not only knowing that SOMETHING happened when you look at your performance metrics graphs, but also exactly WHAT happened by having immediate access to relevant logs right there!

Screenshot – Dashboard with SPM Performance Graphs & Logsene Log Graphs  [click to enlarge]

test_dashboard_SPM_Logsene

Take a Test Drive — It’s Easy and Free to Get Started

Like what you see here?  Sound like something that could benefit your organization?  Then try SPM or Logsene for Free for 30 days by registering here.  There’s no commitment and no credit card required.

Announcement: NGINX and NGINX Plus Monitoring in SPM

The list of monitored stacks keeps growing!  SPM Performance Monitoring, Alerting and Anomaly Detection just added support for NGINX and NGINX Plus.  Now you can capture metrics like total requests and connections along with an overview report and a whole lot more.

NGINX is a high performance, open source web application accelerator that helps millions of the world’s busiest websites deliver more content, faster, to its users.  It is the #1 web server for the top 10k busiest websites in the world. NGINX Plus is the commercial version which adds advanced features, professional services, and shows more juicy metrics.

NGINX and NGINX Plus monitoring in SPM complements recently announced support for apps like MySQL, Cassandra, Memcached, Apache and AWS CloudWatch.  You can see a more complete list here (scroll down to “Monitored Apps”)

Here’s a glimpse into what SPM for NGINX and NGINX Plus provides – click on an image to see the full view or you can look at the actual SPM live demo showing SPM monitoring NGINX and NGINX Plus and their metrics.

Screenshot – NGINX Plus Overview  (click to enlarge)

NGINX+ overview_new

 

Screenshot – NGINX Plus Server Status  (click to enlarge)

NGINX+ server status

 

Screenshot – NGINX Plus Status Zones  (click to enlarge)

NGINX+ status zones 2

 

Screenshot – NGINX Plus Caches  (click to enlarge)

NGINX+ caches

 

Screenshot – NGINX Plus Upstreams  (click to enlarge)

NGINX+ upstreams

 

Screenshot – NGINX Overview  (click to enlarge)

NGINX overview

 

Screenshot – NGINX Status  (click to enlarge)

NGINX status

 

Live Demo — See SPM for Yourself

Check out SPM’s live demo to see NGINX and NGINX Plus monitoring for yourself.  You’ll also be able to poke around and see Kafka, HBase, Elasticsearch, Solr, MySQL, and other types of apps being monitored.

Love the Idea of Monitoring NGINX and NGINX Plus? Take a Test Drive — It’s Easy to Get Started.

Try SPM Performance Monitoring for Free for 30 days by registering here.  There’s no commitment and no credit card required.

Logsene Log Management and Analytics Grows Up!

There are exciting changes on the immediate horizon for Sematext’s comprehensive logging solution: Logsene Log Management and Analytics.  We just announced the seamless integration of Logsene with SPM Performance Monitoring, Alerting and Anomaly Detection (more on that below) and we have a new release of Logsene just weeks away from production.  Here’s a glimpse of the new Logsene UI.  Watch for more log analytics and management goodies on our blog soon.

New Logsene UI sneak peek

logsene1366x768_blog_post

Of course, the new Logsene will keep the option to use the Kibana UI directly from Logsene.

New Logsene Functionality in Words

  1. We support multiple queries in a single view; each query’s data get added to the graph (hence 2 lines in this screenshot).
  2. Queries can be saved, converted to scheduled queries (get email with log graphs every day/week/month type of functionality), or to alert queries (i.e., “email me when we see log messages with XYZ in them”). Those little gray icons next to search fields are for that.
  3. Clicking on the icon to the left of the top search field will show more functionality.
  4. Filtering by various log attributes.
  5. Raw log events in what we internally call Logs Table (or LT). Multiple tabs on top correspond to multiple queries one can enter.
  6. On the left is a list of available fields that are shown in LT. Checking them adds them to LT. They can be dragged and reordered. Reordering them controls the order of columns in LT.
  7. Number behind field names shows the number of distinct values for that field.
  8. Logs can be downloaded as CSV or published to Github Gist or Pastebin services.
  9. Arrows in top-right of LT expand the LT into a “Kiosk mode.”
  10. Little icon in each row in LT can be clicked. It expands on click and lets you see context around the given log.
  11. Clicking on the two gray buttons in top right with down arrows will expose metrics from SPM and Events.

If you currently use Splunk or another log management tool then there is a good chance that you will find Logsene to be an easy and useful way to index, search and analyze your logs.

But Why Use Logsene?

In short: Logsene and SPM provide a single pane of glass for performance monitoring, centralized log management, alerts, anomalies, custom events, and custom KPIs.  Unlike competing products, SPM and Logsene together not only tell users that SOMETHING happened, it now tells them exactly WHAT happened.  No need for a mish-mash of individual open-source or commercial tools for monitoring, for alerting, for logging, for custom dashboards, etc., each with its own configuration, its own UI, etc.

Not only that, but Logsene offers a unique pricing plan that, unlike those of other log management products, does NOT charge for the amount of data and retention.  You only pay for the number of logs in the index – you don’t pay for data transfer or data retention.

We also offer Plan Auto-Upgrade, a unique feature that can automatically switch you to the next higher plan if you exceed your log threshold — preventing you from losing logs which could contain critical information, without forcing you to keep this higher plan.

Moreover, because Logsene usage is “by day”, when auto-upgrade happens the extra cost is really minimal — at any time after the auto-upgrade you can adjust retention and/or plan and go back to the old plan

Check out a Live Demo

See SPM and Logsene (note: old UI and Kibana UI, the new UI will be in the next release) for yourself by viewing a live demo.  You’ll also be able to poke around and see Storm, Kafka, Solr, Elasticsearch, Hadoop, HBase, MySQL, and other types of apps being monitored.

Try Logsene and/or SPM Performance Monitoring for Free for 30 days by registering here.  There’s no commitment and no credit card required.

We’re Hiring!

If you enjoy performance monitoring, log analytics, or search analytics, working with projects like Elasticsearch, Solr, HBase, Hadoop, Kafka, and Storm, then drop us a line.  We’re hiring planet-wide!  Front end and JavaScript Developers, Developer Evangelists, Full-stack Engineers, Mobile App Developers…get in touch!

Monitoring Kafka, Storm and Cassandra Together with SPM

Kafka, Storm and Cassandra — Big Data’s Three Amigos?  Not quite.  And while much less humorous than the movie, this often-used-together trio of tools work closely together to make in-stream processing as smooth, immediate and efficient as possible.  Needless to say, it makes a lot of sense to monitor them together.  And since you’re reading the Sematext blog you shouldn’t be surprised to hear that we offer an all-in-one solution that can be used for Kafka, Storm and Cassandra performance monitoring, alerts and anomaly detectionSPM Performance Monitoring.  Moreover, if you ship logs from any of these three systems to Logsene, you can correlate not only metrics from these three systems, but also their logs and really any other type of event!

Enough with all the words — here is an illustration of how these three tools typically work together:

Kafka-Storm-Cassandra

Of course, you could also be storing data into some other type of data store, like Elasticsearch or HBase or MySQL or Solr or Redis.  SPM monitors all of them, too.

So what do you get if you monitor Kafka, Storm, and Cassandra with SPM?

You get a single pane pane of glass, a single access point through which you can see visualizations of well over 100 metrics you can slice and dice by a number applications-specific dimensions.  For example, various Kafka performance metrics can be filtered by one or more topics, while some Storm performance metrics can be filtered by topology or worker.  Of course, you can set alerts on any of the metrics and you can even make use of SPM’s anomaly detection capabilities.

Kafka, Storm + Cassandra screenshot

 

Consolidate Your App Monitoring, Alerting, and Centralize Logging — It’s Easy!

Many organizations tackle performance monitoring with a mish-mash of different monitoring and alerting tools cobbled together in an uneasy coexistence that is often far from seamless.  SPM takes all that hassle away and makes it easy and comprehensive in one step.

Live Demo — See SPM for Yourself

Check out SPM’s live demo to see this monitoring for yourself.  You won’t find any demo apps showing Cassandra metrics because we don’t use it at Sematext yet, but you’ll be able to poke around and see Kafka, HBase, Elasticsearch, MySQL, and other types of apps being monitored.

Love the Idea of Monitoring Kafka, Storm & Cassandra Together?
Take a Test Drive — It’s Easy to Get Started.

Try SPM Performance Monitoring for Free for 30 days by registering here.  There’s no commitment and no credit card required.

We’re Hiring!

If you enjoy performance monitoring, log analytics, or search analytics, working with projects like Elasticsearch, Solr, HBase, Hadoop, Kafka, and Storm, then drop us a line.  We’re hiring planet-wide!  Front end and JavaScript Developers, Developer Evangelists, Full-stack Engineers, Mobile App Developers…get in touch!

 

Follow

Get every new post delivered to your Inbox.

Join 1,713 other followers