SplunkCloud-9.1.2312-SearchTutorial

1 Search Tutorials > Part 1: Getting started

1.1 Install SW and Demo Data files

1.2 Register the Developer License

2 Part 2: Uploading the tutorial data

2.1 Data Sources

2.2 Upload Data

3 Part 3: Using the Splunk Search App

3.1 Search Time Range

3.1.1 Presets

3.1.2 Relative

3.1.3 Real-time

3.1.4 Date Range

3.1.5 Date & Time Range

3.1.6 Advanced

4 Part 4: Searching the tutorial data

4.1 Basic searches and search results

4.1.1 Patterns, Statistics, and Visualizations

4.2 Use fields to search

4.2.1 access_combined

4.3 Use the search language

4.4 Use a subsearch

5 Part 5: Enriching events with lookups

5.1 Enabling field lookups

5.1.1 Upload the Lookup file

5.1.2 Add the field lookup definition

5.1.2.1 Share the lookup definition with all apps

5.1.3 Make the lookup automatic

5.1.3.1 Share the automatic lookup with all apps

5.2 Search with field lookups

6 Part 6: Creating reports and charts

6.1 Save and share your reports

6.2 Create a basic chart

6.3 Create an overlay chart and explore visualization options

6.4 Create a report from a custom chart

6.5 Create a report from a sparkline chart

7 Part 7: Creating dashboards

7.1 About dashboards

7.2 Create dashboards and panels

7.3 Add more panels to dashboards

1. Search Tutorials > Part 1: Getting started

1. Install SW and Demo Data files 

2. Register the Developer License

http://localhost:8000/

Settings – License – choose the licenseing foder and file.  Once uploaded the license, check http://localhost:8000/en-US/manager/system/licensing/licenses

2. Part 2: Uploading the tutorial data

1. Data Sources

Data source

Description

Files and

directories

Most data that you might be interested in comes directly from files and directories.

Network events

The Splunk software can index remote data from any network port and SNMP events from remote devices.

IT Operations

Data from IT Ops, such as Nagios, NetApp, and Cisco.

Cloud services

Data from Cloud services, such as AWS and Kinesis.

Database services Data from databases such as Oracle, MySQL, and Microsoft SQL Server.

Security services Data from security services such as McAfee, Microsoft Active Directory, and Symantec Endpoint Protection.

Virtualization

services

Data from virtualization services such as VMWare and XenApp.

Application

servers

Data from application servers such as JMX & JMS, WebLogic, and WebSphere.

Windows sources

The Windows version of Splunk software accepts a wide range of Windows-specific inputs, including Windows Event Log,

Windows Registry, WMI, Active Directory, and Performance monitoring.

Other sources

Other input sources are supported, such as FIFO queues and scripted inputs for getting data from APIs, and other remote data interfaces.

2. Upload Data

Settings > Add Data > Upload > tutorialdata.zip file 

Splunk Enterprise for Windows > 

a. Select Regular expression on path.

b. Type \\(.*)\/ for the regex to extract the host values from the path.

Index > Provide index as Tutorialdata – if we choose default it goes to main index and unable to delete if we want

3. Part 3: Using the Splunk Search App

index="tutorialdata" 

index="tutorialdata" buttercupgames

1. Search Time Range 

1. Presets 

2. Relative

3. Real-time

4. Date Range

5. Date & Time Range

6. Advanced

4. Part 4: Searching the tutorial data

• The Search Assistant is more useful after you start learning the search language. When you type search commands, the Search Assistant displays command information.

• In the tutorial the Tutorialdata.zip uploaded with default Index.  But I created index as tutorialdata.  So without Index search will not work becoz without index it search in default or main index.

• The type of search commands that you use determines which tab the search results appear on (in Events, Patterns, Statistics, and Visualization.) 

• 
  

index="tutorialdata"  > 109,864 events

index="tutorialdata" buttercupgames > 36,819 events

index="tutorialdata"  buttercupgames "categoryid=sports"   > 763 events

index="tutorialdata"  "categoryid=sports" > 763 events

index="tutorialdata" buttercupgames (error OR fail* OR severe) > 427 Events, 7 Patterns

• Display Option > Raw, List, Table

• Fields sidebar

• When you add data to the Splunk platform the data is indexed. As part of the index process, information is extracted from your data and formatted as name and value pairs, called fields. When you run a search, the fields are identified and listed in the Fields sidebar next to your search results. The fields are divided into two categories.

• Selected fields 

• are visible in your search results. By default, host, source, and sourcetype appear. You can select other fields to show in your events.

• Interesting fields 

• are other fields that have been extracted from the events in your  earch results.

• ·

1. Basic searches and search results

1. Patterns, Statistics, and Visualizations

The Patterns tab displays a list of the most common patterns among the set of events returned by your search. Each of these patterns represents events that share a similar structure.

The Statistics tab populates when you run a search with transforming commands such as stats, top, chart, and so on.

The keyword search for "buttercupgames" does not show results in this tab because the search does not include any transforming commands.

Searches with transforming commands also populate the Visualization tab. The results area of the Visualizations tab includes a chart and the statistics table that is used to generate the chart.

2. Use fields to search

Search with fields

index="tutorialdata" sourcetype=access_* > 39532 events, 3 Patterns

1. access_combined 

If you are familiar with the access_combined format of Apache logs, you might recognize some of the information in each event, such as:

• IP addresses for the users accessing the website.

• URIs and URLs for the pages requested and referring pages.

• HTTP status codes for each page request.

• GET or POST page request methods.

• To add fields to the Selected Fields list, click All Fields at the top of the Fields sidebar, choose the field.

• 

UC: Search for successful purchases from the Buttercup Games store.

index="tutorialdata" sourcetype=access_* status=200 action=purchase > 5244 events, 2 Paterns

UC: Search for NOT successful purchases from the Buttercup Games store.

index="tutorialdata" sourcetype=access_* status=!200 action=purchase > 0

UC : Search for Errors

index="tutorialdata" (error OR fail* OR severe) OR (status=404 OR status=500 OR status=503) – 36,055

UC : Search for sales of a specific product

index="tutorialdata" sourcetype=access_* status=200 action=purchase  - 5224 (7 categoryid)

index="tutorialdata" sourcetype=access_* status=200 action=purchase categoryId=simulation – 246 

3. Use the search language

• There are two modes for the Search Assistant: Compact and Full.  Default is Compact, You change the Search Assistant mode.

• Administrator > Preferences > SPL Editor – change the setting

UC: Search for successful purchases categorywise count and percentage

index="tutorialdata" sourcetype=access_* status=200 action=purchase | top categoryId   - Under statistics 7 category listed with name, count and percent.  see the picture in URL

4. Use a subsearch 

• how to correlate events by using subsearches.

UC: ind the most frequent shopper without subsearch

index="tutorialdata" sourcetype=access_* status=200 action=purchase | top limit=1 clientip

• how many different products the VIP shopper has purchased using stats

index="tutorialdata" sourcetype=access_* status=200 action=purchase clientip=87.194.216.51 | stats count, distinct_count(productId), values(productId) by clientip

For all clientIP

index="tutorialdata" sourcetype=access_* status=200 action=purchase | stats count, distinct_count(productId), values(productId) by clientip

UC: ind the most frequent shopper using subsearch

• identify the single most frequent shopper

index="tutorialdata" sourcetype=access_* status=200 action=purchase | top limit=1 clientip | table clientip

Table shows only client ip

index="tutorialdata" sourcetype=access_* status=200 action=purchase | top limit=1 clientip > this shows ip, count, %

index="tutorialdata" sourcetype=access_* status=200 action=purchase [search index="tutorialdata"  sourcetype=access_* status=200 action=purchase | top limit=1 clientip | table clientip] | stats count, distinct_count(productId), values(productId) by clientip

Make the search syntax easier to read > Ctrl + \

index="tutorialdata" sourcetype=access_* status=200 action=purchase 

    [ search index="tutorialdata" sourcetype=access_* status=200 action=purchase 

    | top limit=2 clientip 

    | table clientip] 

| stats count, distinct_count(productId), values(productId) by clientip

You can make the information more understandable by renaming the columns

index="tutorialdata" sourcetype=access_* status=200 action=purchase 

    [search index="tutorialdata" sourcetype=access_* status=200 action=purchase 

    | top limit=1 clientip 

    | table clientip] 

| stats count AS "Total Purchased", distinct_count(productId) AS "Total Products", values(productId) AS "Product IDs" by clientip 

| rename clientip AS "VIP Customer"

5. Part 5: Enriching events with lookups

1. Enabling field lookups 

There are five key steps to enabling field lookups:

1. Upload the lookup file

2. Share the uploaded file with the applications

3. Create a lookup definition

4. Share the lookup definition with the applications

5. Optional. Make the lookup definition automatic

Download the Prices.csv.zip file. Uncompress the Prices.csv.zip file. There is only one file in the ZIP file, prices.csv. The prices.csv file contains the product names, price, and code.

1. Upload the Lookup file

Settings > Knowledge > Lookups > Lookup files > Add New

 

• Share the Lookup files to all apps 

• Go to Lookup table files  > choose the Path of prices.csv > under sharing click the permissions > Choose All apps and all users

• 

2. Add the field lookup definition

Settings > Knowledge > Lookups > Lookup definitions > Add new

1.  Share the lookup definition with all apps

Settings > Knowledge > Lookups > Lookup definitions > choose the newly added lookup definition > Permission > All Apps

• You can use this field lookup to add information from the lookup table file to your events. You use the field lookup by specifying the lookup command in a search. Or, you can set the field lookup to run automatically.

• 
  

3. Make the lookup automatic 

Settings > Knowledge > Lookups > Automatic lookups > New Automatic Lookup

Provide the inputs

Search data file contains ProductId but no Name, prie, sales_price

Automatic Lookup – Add New 

1. Share the automatic lookup with all apps 

Permission – share to all 

2. Search with field lookups

• Because the prices_lookup is an automatic lookup, the fields from the lookup table will automatically appear in your search results.

• When you setup the automatic lookup, you specified that the productId field in your indexed events corresponds to the productId field in the prices.csv file.

• index="tutorialdata" sourcetype=access_*

• moves the prices, productName  fields from the list of Interesting Fields to the list of Selected Fields in the Fields sidebar.

index="tutorialdata" sourcetype=access_* 

| stats values(price) BY productname

• Display the VIP client purchases

index="tutorialdata" sourcetype=access_* status=200 action=purchase 

    [ search index="tutorialdata" sourcetype=access_* status=200 action=purchase 

    | top limit=1 clientip 

    | table clientip] 

| stats count AS "Total Purchased", dc(productId) AS "Total Products", values(productId) AS "Product IDs" BY clientip 

| rename clientip AS "VIP Customer"

• With product names

index="tutorialdata" sourcetype=access_* status=200 action=purchase 

    [ search index="tutorialdata" sourcetype=access_* status=200 action=purchase 

    | top limit=1 clientip 

    | table clientip] 

| stats count AS "Total Purchased", dc(productId) AS "Total Products", values(productname) AS "Product Names" BY clientip 

| rename clientip AS "VIP Customer"

6. Part 6: Creating reports and charts

1. Save and share your reports

index="tutorialdata" sourcetype=access_* status=200 action=purchase 

    [ search index="tutorialdata" sourcetype=access_* status=200 action=purchase 

    | top limit=1 clientip 

    | table clientip] 

| stats count AS "Total Purchased", dc(productId) AS "Total Products", values(productname) AS "Product Names" BY clientip 

| rename clientip AS "VIP Customer"

• Search the above query – after the result > Save As > Report 

2. Create a basic chart

index="tutorialdata" sourcetype=access_* status=200 

| chart count AS views count(eval(action="addtocart")) AS addtocart count(eval(action="purchase")) AS purchases by productname 

| rename productname AS "Product Name", views AS "Views", addtocart AS "Adds to Cart", purchases AS "Purchases"

3. Create an overlay chart and explore visualization options

• Calculated fields 

index="tutorialdata" sourcetype=access_* status=200 

| stats count AS views count(eval(action="addtocart")) AS addtocart count(eval(action="purchase")) AS purchases by productname 

| eval viewsToPurchases=(purchases/views)*100 

| eval cartToPurchases=(purchases/addtocart)*100 

| table productname views addtocart purchases viewsToPurchases cartToPurchases 

| rename productname AS "Product Name", views AS "Views", addtocart as "Adds To Cart", purchases AS "Purchases"

4. Create a report from a custom chart

index="tutorialdata"sourcetype=access_* | timechart count(eval(action="purchase")) by productname usenull=f useother=f

without eval all values shows 0 – ???

5. Create a report from a sparkline chart

index="tutorialdata" sourcetype=access_* status=200 action=purchase| chart sparkline(count) AS "Purchases Trend" count AS Total BY categoryId | rename categoryId AS Category

7. Part 7: Creating dashboards

1. About dashboards

• Dashboards are views that are made up of panels. The panels can contain modules such as search boxes, fields, charts, tables, and lists. Dashboard panels are usually connected to reports.

• 
  

2. Create dashboards and panels

3. Add more panels to dashboards