JOB: Devops Evangelist – Monitoring, Logging, Analytics

DEVELOPER / DEVOPS EVANGELIST

Sematext is looking for someone with a business and marketing bent and enough technical background to be able to put together bits of code and demos of SPM, Logsene, and other products we are working on.  A good fit for this role is a person who likes to teach and share, knows how to connect with people and their needs, is passionate and is considered (or wants to become) a thought leader in at least one area — Monitoring, Logging, Data Analytics and/or Business Intelligence.  Our ideal evangelist also enjoys the agility and challenge of a startup.  For a good description of the type of person we are looking for watch this video.  Sematext is a young, fast growing, highly distributed and agile team and our developer evangelist will work in many different capacities and contribute to the company’s success in a variety of ways.

 

RESPONSIBILITIES:

  • Create technical content and demos for publication on our blog and other channels to show developers, devops, and others how to implement specific solutions or use new technologies
  • Prepare and deliver presentations and webinars, speak at industry conferences, local meetups and other events
  • Build relationships with tech bloggers, open source contributors and product community leaders, journalists and analysts
  • Educate and empower developers, giving technical workshops and brown bags
  • Build partnerships with individuals, companies and organizations that serve the same communities we do (Elasticsearch, Solr, Kafka, Hadoop, Storm, Spark, etc.)
  • Gather and socialize product feedback that informs engineering, sales, and marketing decision making

 

REQUIREMENTS:

  • BS or higher in Computer Science or professional experience as a developer, sys admin, sales engineer or other technical role
  • Strong verbal and written communications skills with ability to write for engineers or high-level management
  • Entrepreneurial thinking and the ability to act effectively with only high-level direction

 

BONUS:

  • Participation in open-source community
  • Experience with other commercial and open source monitoring, logging, or analytics technologies
  • Experience working in a startup

 

You can check out our products, our services, our clients, and our team to get a better sense of what Sematext is all about.  Also worth a look are the 19 things you may like about Sematext.  Interested? Please send your resume to jobs@sematext.com.  For other job openings please see Jobs @ Sematext or even our previous job listings.

 

Logsene Log Management and Analytics Grows Up!

There are exciting changes on the immediate horizon for Sematext’s comprehensive logging solution: Logsene Log Management and Analytics.  We just announced the seamless integration of Logsene with SPM Performance Monitoring, Alerting and Anomaly Detection (more on that below) and we have a new release of Logsene just weeks away from production.  Here’s a glimpse of the new Logsene UI.  Watch for more log analytics and management goodies on our blog soon.

New Logsene UI sneak peek

logsene1366x768_blog_post

Of course, the new Logsene will keep the option to use the Kibana UI directly from Logsene.

New Logsene Functionality in Words

  1. We support multiple queries in a single view; each query’s data get added to the graph (hence 2 lines in this screenshot).
  2. Queries can be saved, converted to scheduled queries (get email with log graphs every day/week/month type of functionality), or to alert queries (i.e., “email me when we see log messages with XYZ in them”). Those little gray icons next to search fields are for that.
  3. Clicking on the icon to the left of the top search field will show more functionality.
  4. Filtering by various log attributes.
  5. Raw log events in what we internally call Logs Table (or LT). Multiple tabs on top correspond to multiple queries one can enter.
  6. On the left is a list of available fields that are shown in LT. Checking them adds them to LT. They can be dragged and reordered. Reordering them controls the order of columns in LT.
  7. Number behind field names shows the number of distinct values for that field.
  8. Logs can be downloaded as CSV or published to Github Gist or Pastebin services.
  9. Arrows in top-right of LT expand the LT into a “Kiosk mode.”
  10. Little icon in each row in LT can be clicked. It expands on click and lets you see context around the given log.
  11. Clicking on the two gray buttons in top right with down arrows will expose metrics from SPM and Events.

If you currently use Splunk or another log management tool then there is a good chance that you will find Logsene to be an easy and useful way to index, search and analyze your logs.

But Why Use Logsene?

In short: Logsene and SPM provide a single pane of glass for performance monitoring, centralized log management, alerts, anomalies, custom events, and custom KPIs.  Unlike competing products, SPM and Logsene together not only tell users that SOMETHING happened, it now tells them exactly WHAT happened.  No need for a mish-mash of individual open-source or commercial tools for monitoring, for alerting, for logging, for custom dashboards, etc., each with its own configuration, its own UI, etc.

Not only that, but Logsene offers a unique pricing plan that, unlike those of other log management products, does NOT charge for the amount of data and retention.  You only pay for the number of logs in the index – you don’t pay for data transfer or data retention.

We also offer Plan Auto-Upgrade, a unique feature that can automatically switch you to the next higher plan if you exceed your log threshold — preventing you from losing logs which could contain critical information, without forcing you to keep this higher plan.

Moreover, because Logsene usage is “by day”, when auto-upgrade happens the extra cost is really minimal — at any time after the auto-upgrade you can adjust retention and/or plan and go back to the old plan

Check out a Live Demo

See SPM and Logsene (note: old UI and Kibana UI, the new UI will be in the next release) for yourself by viewing a live demo.  You’ll also be able to poke around and see Storm, Kafka, Solr, Elasticsearch, Hadoop, HBase, MySQL, and other types of apps being monitored.

Try Logsene and/or SPM Performance Monitoring for Free for 30 days by registering here.  There’s no commitment and no credit card required.

We’re Hiring!

If you enjoy performance monitoring, log analytics, or search analytics, working with projects like Elasticsearch, Solr, HBase, Hadoop, Kafka, and Storm, then drop us a line.  We’re hiring planet-wide!  Front end and JavaScript Developers, Developer Evangelists, Full-stack Engineers, Mobile App Developers…get in touch!

Monitoring Kafka, Storm and Cassandra Together with SPM

Kafka, Storm and Cassandra — Big Data’s Three Amigos?  Not quite.  And while much less humorous than the movie, this often-used-together trio of tools work closely together to make in-stream processing as smooth, immediate and efficient as possible.  Needless to say, it makes a lot of sense to monitor them together.  And since you’re reading the Sematext blog you shouldn’t be surprised to hear that we offer an all-in-one solution that can be used for Kafka, Storm and Cassandra performance monitoring, alerts and anomaly detectionSPM Performance Monitoring.  Moreover, if you ship logs from any of these three systems to Logsene, you can correlate not only metrics from these three systems, but also their logs and really any other type of event!

Enough with all the words — here is an illustration of how these three tools typically work together:

Kafka-Storm-Cassandra

Of course, you could also be storing data into some other type of data store, like Elasticsearch or HBase or MySQL or Solr or Redis.  SPM monitors all of them, too.

So what do you get if you monitor Kafka, Storm, and Cassandra with SPM?

You get a single pane pane of glass, a single access point through which you can see visualizations of well over 100 metrics you can slice and dice by a number applications-specific dimensions.  For example, various Kafka performance metrics can be filtered by one or more topics, while some Storm performance metrics can be filtered by topology or worker.  Of course, you can set alerts on any of the metrics and you can even make use of SPM’s anomaly detection capabilities.

Kafka, Storm + Cassandra screenshot

 

Consolidate Your App Monitoring, Alerting, and Centralize Logging — It’s Easy!

Many organizations tackle performance monitoring with a mish-mash of different monitoring and alerting tools cobbled together in an uneasy coexistence that is often far from seamless.  SPM takes all that hassle away and makes it easy and comprehensive in one step.

Live Demo — See SPM for Yourself

Check out SPM’s live demo to see this monitoring for yourself.  You won’t find any demo apps showing Cassandra metrics because we don’t use it at Sematext yet, but you’ll be able to poke around and see Kafka, HBase, Elasticsearch, MySQL, and other types of apps being monitored.

Love the Idea of Monitoring Kafka, Storm & Cassandra Together?
Take a Test Drive — It’s Easy to Get Started.

Try SPM Performance Monitoring for Free for 30 days by registering here.  There’s no commitment and no credit card required.

We’re Hiring!

If you enjoy performance monitoring, log analytics, or search analytics, working with projects like Elasticsearch, Solr, HBase, Hadoop, Kafka, and Storm, then drop us a line.  We’re hiring planet-wide!  Front end and JavaScript Developers, Developer Evangelists, Full-stack Engineers, Mobile App Developers…get in touch!

 

Announcement: What’s New in SPM Performance Monitoring

A new SPM Performance Monitoring release was just pushed to production and it’s chock full of great new stuff to complement its proactive performance monitoring, alerting, anomaly detection, etc., available in the Cloud or On Premise.  Here is a run-down of the juicier additions. The slightly longer version can be found in the SPM Changelog.

Integration with Logsene Log Management and Analytics

SPM performance monitoring now gives users access to even more metrics by seamlessly integrating with event data and logs via Logsene Log Management and Analytics.  This enables correlation across performance metrics, alerts, anomaliesevents, logs, and provides a single pane of glass across any organization’s IT infrastructure.

Monitoring Support for More Applications

We’ve added native monitoring support for the following applications to complement monitoring for Solr, Elasticsearch, Hadoop, HBase, Storm, Redis, Kafka, ZooKeeper and many others.

Screenshots

Eager to see pictures instead of reading content?  Then jump below to see screenshots of these apps being monitored.

UI/UX Improvements

UI/UX improvements include: zooming and panning, client-side caching, wider and simpler metric charts, new filter slide-out panels with search capabilities, quick access to all dashboards and easier dashboard creation, and more.

Event Graphs

Events and event graphs are now integrated into SPM Performance Monitoring.  You can now correlate various types of events, such as alerts, anomalies, application deployments, restarts, releases, server reboots, etc., with performance metrics graphs, as well as with logs and log graphs.  Many of you will also be happy to hear that SPM can now turn Alerts into Events, and graph them as well.  Check out Event Integration if you want to publish your own Events.

More Powerful Storm Monitoring

SPM Storm monitoring now serves up more metrics, more graphs and provides more granular details.  This includes the addition of metric filters and the ability to monitor not just Spouts and Bolts, but also monitor Storm Workers.

Dashboard Enhancements

Creating and working with dashboards just got a lot more intuitive and flexible.  This includes:

  • creating new dashboards via an intuitive “build your own dashboard” tool
  • easier navigation via Miller Columns (think column-oriented view in OSX Finder)
  • adding whatever graphs you want to an existing or brand new dashboard from within that dashboard
  • a pull down menu to select specific dashboards for much quicker access to a specific dashboard

 

Screenshot – SPM Dashboard (one of many possible views; click to enlarge)

Dashboard_test

 

Screenshot – Cassandra Overview  (click to enlarge)

cassandra_overview

 

Screenshot – MySQL Overview  (click to enlarge)

MySQL Overview

Screenshot – Memcached Overview  (click to enlarge)

memcached-overview

Screenshot – Apache Monitoring Overview  (click to enlarge)

Apache Overview

 

Screenshot – AWS CloudWatch EBS Read/Write Bandwidth  (click to enlarge)

AWS_EBS Read:Write Bandwidth

 

Live Demo

Check out SPM’s live demo to see it for yourself.  You won’t find any demo apps showing Cassandra or Memcached metrics because we don’t use them at Sematext yet, but you’ll be able to poke around and see other types of apps being monitored — like Solr, Kafka, Hadoop and HBase, for example — along with MySQL, AWS, and Apache.

Consolidate Your App Monitoring — It’s Easy!

Many organizations tackle performance monitoring with a mish-mash of different monitoring and alerting tools cobbled together in an uneasy coexistence that is often far from seamless. SPM takes all that hassle away and makes it easy and comprehensive in one step.

Try SPM for Free for 30 Days

Try SPM Performance Monitoring for Free for 30 days by registering here.  There’s no commitment and no credit card required.

We’re Hiring!

If you enjoy performance monitoring, log analytics, or search analytics, working with projects like Elasticsearch, Solr, HBase, Hadoop, Kafka, and Storm, then drop us a line.  We’re hiring planet-wide!  Front end and JavaScript Developers, Developer Evangelists, Full-stack Engineers, Mobile App Developers…get in touch!

Announcement: Cassandra Performance Monitoring in SPM

Cassandra is a distributed database management system built to handle massive data sets while providing high availability without compromising performance.  That’s why many organizations use it for their mission-critical data. That being said, if you are running Cassandra then you’ll want to keep close tabs on it.  And now SPM Performance Monitoring can help you do just that as the latest release supports Cassandra performance monitoring.  Not to mention that all the usual SPM alerting, anomaly detection, etc., can be used with any of the Cassandra metrics.

Why Not Monitor More Than Just Cassandra?

Unlike some of the tools that only monitor Cassandra and nothing else, SPM Performance Monitoring covers Cassandra, the competing database HBase, and a lot more: Solr, Elasticsearch, Hadoop, MySQL, AWS CloudWatch, Memcached, Apache, and just about any other app you want to monitor.  SPM also monitors Storm and Kafka, which are often used together with Cassandra (considered their most popular data store).  Now all three can be monitored together!

Have a look at a few of the screenshots to see how we graph Cassandra metrics in SPM (a list of Cassandra metrics we monitor is listed further below).  You can also check out SPM’s live demo. You won’t find any demo apps showing Cassandra metrics because we don’t use Cassandra at Sematext yet, but you’ll be able to poke around and see other types of apps being monitored, like Solr, Kafka, Hadoop and HBase, for example.

Overview  (click to enlarge)

cassandra_overview

 

Compactions  (click to enlarge)

compactions

 

Pending Writes  (click to enlarge)

pending-writes

 

Write Requests  (click to enlarge)

write-requests

 

SSTable  (click to enlarge)

sstable

 

Bloom Filter  (click to enlarge)

bloom-filter

SPM now monitors Cassandra metrics like:

  • Write Requests (rate, count, latency)
  • Read Requests (rate, count, latency)
  • Pending Write Operations (flushes, post flushes, write requests, replication of write)
  • Pending Read Operations (read requests, read repair tasks, compactions)
  • Pending Cluster Operations (manual repair tasks, gossip tasks, hinterd handoff, internal responses, migrations, misc tasks, request responses)
  • Compactions
  • Row/Key Cache (hit ratio, requests count)
  • Local Writes (rate, count, latency)
  • Local Reads (rate, count, latency)
  • SSTable (size, count)
  • Bloom Filter (space used, false positives ratio)

Please tell us what you think – @sematext is always listening!  Is there something SPM Performance Monitoring doesn’t monitor that you would really like to monitor?

Consolidate Your App Monitoring — It’s Easy!

Many organizations tackle performance monitoring with a mish-mash of different monitoring and alerting tools cobbled together in an uneasy coexistence that is often far from seamless.  Think Graphite+Nagios, for example.  SPM takes all that hassle away and makes it easy and comprehensive in one step.

Try SPM for Free for 30 Days

Try SPM Performance Monitoring for Free for 30 days by registering here.  There’s no commitment and no credit card required.

We’re Hiring

If you enjoy performance monitoring, log analytics, or search analytics, working with projects like Elasticsearch, Solr, HBase, Hadoop, Kafka, and Storm, then drop us a line.  We’re hiring planet-wide!  Front end and JavaScript Developers, Developer Evangelists, Full-stack Engineers, Mobile App Developers…get in touch!

Announcement: Apache Monitoring in SPM

Responsible for your organization’s Apache servers?  If so, you’ll be happy to hear that you can now capture metrics about your Apache web servers — along with alerting and anomaly detection, traffic spikes, dips/gaps, etc. — with SPM Performance Monitoring.  This includes system-wide visibility into resource utilization, application performance, and operational health.

We’re also announcing other great new additions to SPM in the coming days to complement just-released monitoring support for MySQL, Cassandra, Memcached and AWS CloudWatch.  Watch this space for details…

Here’s a glimpse into what SPM for Apache provides – click on an image to see the full view or you can look at the actual SPM live demo showing some of our own Apache servers and their metrics.

Overview  (click to enlarge)

Apache Overview

 

Workers  (click to enlarge)

Apache Workers

 

Traffic Rate  (click to enlarge)

Apache Traffic Rate

 

Scoreboard Serving  (click to enlarge)

Apache Scoreboard Serving

 

CPU Details  (click to enlarge)

Apache CPU Details

 

Memory Details  (click to enlarge)

Apache Memory Details

 

IO Read/Write  (click to enlarge)

Apache IO Read:Write

 

Network Traffic  (click to enlarge)

Apache Network Traffic

 

Please tell us what you think – @sematext is always listening!  Is there something SPM Performance Monitoring doesn’t monitor that you would really like to monitor?

Why Have Something to Just Monitor Apache?

…When you can monitor almost everything with one solution — SPM.  Many organizations tackle performance monitoring with a mish-mash of different monitoring and alerting tools cobbled together in an uneasy coexistence that is often far from seamless.  Think Graphite and Nagios just for Apache, and numerous other tools for…everything else.  SPM takes all that hassle away and makes it easy and comprehensive in one step.

Try SPM for Free for 30 Days

Try SPM Performance Monitoring for Free for 30 days by registering here.  There’s no commitment and no credit card required.

We’re Hiring!

If you enjoy performance monitoring, log analytics, or search analytics, working with projects like Elasticsearch, Solr, HBase, Hadoop, Kafka, and Storm, then drop us a line.  We’re hiring planet-wide!  Front end and JavaScript Developers, Developer Evangelists, Full-stack Engineers, Mobile App Developers…get in touch!

Announcement: AWS CloudWatch Metrics in SPM

Wouldn’t it be great to have metrics for your AWS resources captured in one place?  And beyond just capturing them, to also have the ability to do useful things with those metrics like create custom dashboards, detect and get alerted on metric anomalies, correlate them with other application events and logs, etc., all in a single pane of glass?  Well…you can!  SPM Performance Monitoring Alerting and Anomaly Detection now captures metrics about your AWS resources via AWS CloudWatch.  This includes system-wide visibility into resource utilization, application performance, and operational health.

Why is this important?

AWS shows metrics for various AWS resources in CloudWatch available via AWS Management Console.  This is nice, but it is not very practical if you already use and prefer SPM for your non-AWS resources (e.g. servers and applications running in your data center) or if you are already shipping your logs to Logsene.  Do you really want to use another, separate UI for monitoring just your AWS resources?  It’s also not practical to use alerting in CloudWatch if you already use alerting and anomaly detection functionality in SPM. Now that SPM gathers metrics for your AWS resources you can have a single place to see all your metrics, alerts, and anomalies.

Today we are exposing all Elastic Cloud Compute (EC2), Elastic Load Balancer (ELB), and Elastic Block Store (EBS) metrics in SPM.  We will continue to add other AWS services to this list.  Having AWS metrics in SPM means that you can apply not only threshold-based alerting to your AWS metrics, which AWS itself provides, but also SPM Anomaly Detection which is much more useful and which AWS CloudWatch does not offer.

Headache-relieving and Time-saving Benefits for Your Organization

Major benefits to using SPM to monitor AWS CloudWatch include:

  • there is nothing to install (i.e., it’s all agentless)
  • AWS cost and performance optimization
  • increase in transparency (i.e., now that AWS metrics are in a common monitoring app more people in your organization can see what you have running in AWS)

Have a look at a few of the screenshots to see some of the AWS metrics SPM graphs.  You can also check out SPM’s live demo. Or, if you prefer to see the full list of AWS metrics SPM captures, just jump down past the screenshots to see them listed below.

EBS Read/Write Bandwidth  (click to enlarge)

AWS_EBS Read:Write Bandwidth

 

EBS Read/Write Latency  (click to enlarge)

AWS_EBS Read:Write Latency

 

EC2 CPU Utilization  (click to enlarge)

AWS_EC2 CPU Utilization

 

EC2 Read/Write Operations per Second  (click to enlarge)

AWS_EC2 Read:Write Operations

 

EC2 Network In/Out  (click to enlarge)

AWS_EC2 Network In:Out

 

ELB Backend 2XX/3XX/4XX/5XX Response Counts  (click to enlarge)

AWS_ELB Backend Responses

 

ELB Healthy and Unhealthy Instance Counts  (click to enlarge)

AWS_ELB Healthy:Unhealthy Instances

 

ELB Request Count  (click to enlarge)

AWS_ELB Requests Count

AWS Metrics List

Here is the complete list of AWS metrics that SPM gathers as of today:

EC2:

  • CPU Utilization
  • Disk Read Operations
  • Disk Write Operations
  • Disk Read Bytes
  • Disk Write Bytes
  • Network In
  • Network Out
  • Status Check Failed
  • Status Check Failed (Instance)
  • Status Check Failed (System)

ELB:

  • Healthy Host Count
  • UnHealthy Host Count
  • Request Count
  • Latency
  • ELB 4XX Responses Count
  • ELB 5XX Responses Count
  • Backend 2XX Responses Count
  • Backend 3XX Responses Count
  • Backend 4XX Responses Count
  • Backend 5XX Responses Count
  • Backend Connection Errors Count
  • Surge Queue Length
  • Spillover Count

EBS:

  • Volume Read Bytes
  • Volume Write Bytes
  • Volume Read Ops
  • Volume Write Ops
  • Volume Total Read Time
  • Volume Total Write Time
  • Volume Idle Time
  • Volume Queue Length

Metrics available for IOPS provisioned instances:

  • VolumeThroughputPercentage
  • VolumeConsumedReadWriteOps

Please tell us what you think – @sematext is always listening!  Is there something SPM Performance Monitoring doesn’t monitor that you would really like to monitor?

Try SPM for Free for 30 Days

Try SPM Performance Monitoring for Free for 30 days by registering here.  There’s no commitment and no credit card required.

We’re Hiring!

If you enjoy performance monitoring, log analytics, or search analytics, working with projects like Elasticsearch, Solr, HBase, Hadoop, Kafka, and Storm, then drop us a line.  We’re hiring planet-wide!  Front end and JavaScript Developers, Developer Evangelists, Full-stack Engineers, Mobile App Developers…get in touch!

Follow

Get every new post delivered to your Inbox.

Join 1,652 other followers