Video and Presentation: Indexing and Searching Logs with Elasticsearch or Solr

Interested in log indexing using Elasticsearch or Solr?  Also interested in searching and analyzing logs in real time?

This topic really hits home for us since we released our log analytics tool, Logsene and we also offer consulting services for logging infrastructure.  If you are reading this and looking for a new opportunity then you might be interested to hear that we are hiring worldwide.

If you are into logging like we are, then you will want to check out this presentation delivered by Sematext’s own Radu Gheorghe to the NYC Search, Discovery and Analytics Meetup held recently at Pivotal Labs.  For the purposes of this presentation the term “logs” ranges from server logs and application events to metrics and even social media information.

The presentation has three parts:

  1. Overview of logging tools that play nicely with Elasticseach and Solr (like Logstash, Apache Flume or rsyslog)
  2. Performance tuning and scaling Elasticsearch and Solr
  3. Demo of an end-to-end solution

Here you go – enjoy!

JOB: Professional Services Lead – Solr and Elasticsearch

We have a great opportunity at Sematext for a person who wants to take the Professional Services Lead role and grow both him/herself in this role as well as grow the whole Professional Services side of the house.  The person in this role will get to learn all aspects of the business from engineering, to speaking with numerous clients and customers, to working with remote team members, even touching on sales and marketing.  This position offers a truly multifaceted view into Sematext and the space that Sematext is in, which is a rich blend of search, big data, analytics, open source, products, services, engineering, support, etc.  The ideal candidate would already be in New York, where Sematext HQ is located, but we are open to people from other locations as well.

REQUIREMENTS
• Experience working with Solr or Elasticsearch
• Plan and coordinate customer engagements from business and technical perspective
• Identify customer pain points, needs, and success criteria at the onset of each engagement
• Provide expert-level consulting and support services and strive to be a trustworthy advisor to a wide range of customers
• Resolve complex search issues involving Solr or Elasticsearch
• Identify opportunities to provide customers with additional value through our products or services
• Communicate high-value use cases and customer feedback to our Product teams
• Participate in open source community by contributing bug fixes, improvements, answering questions, etc.

EXPERIENCE
• BS or higher in Engineering or Computer Science preferred
• 2 or more years of IT Consulting and/or Professional Services experience required
• Exposure to other related open source projects (Hadoop, Nutch, Kafka, Storm, Mahout, etc.) a plus
• Experience with other commercial and open source search technologies a plus
• Enterprise Search, eCommerce, and/or Business Intelligence experience a plus
• Experience working in a startup a plus

Interested? Please send your resume to jobs@sematext.com.

For other job openings please see Jobs @ Sematext or even our previous job listings.

Meetup: Indexing and Searching Logs with Elasticsearch and Solr

If you are into logging and search like we are, and if you are in New York, like some of us are, come to Indexing and Searching Logs with Elasticsearch and Solr on Wednesday at Pivotal Labs office in Manhattan.

Video: Scaling Solr with SolrCloud

During last  year’s Lucene Revolution conference in Dublin we had the opportunity to give four talks, one of which was Scaling Solr with SolrCloud. Through it we wanted to share our experiences around scaling Solr, especially as we have experience in running Solr internally and as a team of search consultants.  Enjoy the video and/or the slides!

Note: we are looking for engineers passionate about search to join our professional services team.  We’re hiring planet-wide!

Video: Administering and Monitoring SolrCloud Clusters

As you know, at Sematext, we are not only about consulting services, but also about administration, monitoring, and data analysis. Because of that, during last year’s Lucene Revolution conference in Dublin we gave a talk about administration and monitoring of SolrCloud clusters. During the talk, Rafał Kuć discusses some administration procedures for SolrCloud like collection management and schema modifications with the schema API. In addition, he also talks about why monitoring is important and what to pay attention to. Finally, he shows three real life examples of monitoring usefulnesses.  Enjoy the video and/or the slides!

Note: we are looking for engineers passionate about search to join our professional services team.  We’re hiring planet-wide!

 

 

 

Video: Using Solr for Logs with Rsyslog, Flume, Fluentd and Logstash

A while ago we published the slides from our talk at Lucene Revolution about using Solr for indexing and searching logs. This topic is of special interest for us, since we’ve released Logsene and we’re also offering consulting services for logging infrastructure. If you’re also into working with search engines or logs, please note that we’re hiring worldwide.

The video for that talk is now available, and you can watch it below. The talk is made of three parts:

  • one that discusses the general concepts of what a log is, structured logging and indexing logs in general, whether it’s Solr or Elasticsearch
  • one that shows how to use existing tools to send logs to Solr: Rsyslog and Fluentd to send structured events (yes, structured syslog!); Apache Flume and Logstash to take unstructured data, make it structured via Morphlines and Grok, and then send it to Solr
  • one that shows how to optimize Solr’s performance for handling logs. From tuning the commit frequency and merge factor to using time-based collections with aliases

Poll: Using SolrCloud or Not?

It’s been 9 months since we conducted a poll on SolrCloud usage.  A lot of things can change in 9 months.  SolrCloud itself went through a ton of development and bug fixing since our last poll.  It’s time to see how many of us are using SolrCloud now, at the end of 2013.

Please tweet this poll and help us spread the word, so we can get a good, statistically significant results.

Follow

Get every new post delivered to your Inbox.

Join 1,683 other followers