%pip install elasticsearch jsonlinesQuery
Elasticsearch (Source: technocratsid.com)
Remember when we learned to use Elasticsearch’s Search API to retrieve data from an Index? But what if we want to fine-tune our search? Well, that’s where queries come in. Queries are the precise tools we can use with the Search API to hone in on the data we’re after.
Before we start practicing, install the elasticsearch package
import the packages we need
from elasticsearch import Elasticsearch, helpers
import time
import json
import jsonlinesCreate a connection to elasticsearch, make sure elasticsearch is running on your computer’s localhost or is running on Google Collab.
es = Elasticsearch([{'host': 'localhost', 'port': 9200, 'scheme': 'http'}])Creating an Customer Index
# Create index customers
# PUT http://localhost:9200/customers
response = es.indices.create(index='customers', ignore=400)
print(json.dumps(response.body, indent=4))C:\Users\aditi\AppData\Local\Temp\ipykernel_35632\1248388089.py:3: DeprecationWarning: Passing transport options in the API method is deprecated. Use 'Elasticsearch.options()' instead.
response = es.indices.create(index='customers', ignore=400){
"acknowledged": true,
"shards_acknowledged": true,
"index": "customers"
}Insert Customer Data with Hobbies and Banks
# bulk insert customers
# POST http://localhost:9200/_bulk
# < customers.json
def load_json(file_name):
with jsonlines.open(file_name) as reader:
for obj in reader:
yield {
"_index": "customers",
"_source": obj,
}
helpers.bulk(es, load_json('https://storage.googleapis.com/rg-ai-bootcamp/database/customers-with-banks.json'))(2000, [])Match All
When using Elasticsearch’s Search API, you need a way to retrieve all data from an index without specifying any query. By default, the Search API uses the Match All query. This query is designed to retrieve all data in an index, providing a simple and efficient way to obtain a complete set of data.
# search using match_all
# POST http://localhost:9200/customers/_search
response = es.search(index='customers', body={
'query': {
'match_all': {}
}
})
print(json.dumps(response.body, indent=4)){
"took": 3,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2000,
"relation": "eq"
},
"max_score": 1.0,
"hits": [
{
"_index": "customers",
"_id": "KbJGYowBtsOVXdO2r74I",
"_score": 1.0,
"_source": {
"index": {
"_index": "customers",
"_id": "username1"
}
}
},
{
"_index": "customers",
"_id": "KrJGYowBtsOVXdO2r74I",
"_score": 1.0,
"_source": {
"username": "username1",
"first_name": "Rollie",
"last_name": "Farge",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1984-11-23",
"address": {
"street": "227 Eastwood Pass",
"city": "New York City",
"province": "New York",
"country": "United States",
"zip_code": "10131"
},
"hobbies": [
"Coding",
"Gaming"
],
"banks": [
{
"name": "Mandiri",
"account_number": 8949575
},
{
"name": "Mandiri",
"account_number": 9256376
},
{
"name": "Mandiri",
"account_number": 7904606
}
]
}
},
{
"_index": "customers",
"_id": "K7JGYowBtsOVXdO2r74I",
"_score": 1.0,
"_source": {
"index": {
"_index": "customers",
"_id": "username2"
}
}
},
{
"_index": "customers",
"_id": "LLJGYowBtsOVXdO2r74I",
"_score": 1.0,
"_source": {
"username": "username2",
"first_name": "Toinette",
"last_name": "Ketteridge",
"email": "[email protected]",
"gender": "Female",
"birth_date": "2000-06-07",
"address": {
"street": "48 Golf View Point",
"city": "Youngstown",
"province": "Ohio",
"country": "United States",
"zip_code": "44505"
},
"hobbies": [
"Reading",
"Coding"
],
"banks": [
{
"name": "BNI",
"account_number": 7051376
},
{
"name": "BNI",
"account_number": 9284273
}
]
}
},
{
"_index": "customers",
"_id": "LbJGYowBtsOVXdO2r74I",
"_score": 1.0,
"_source": {
"index": {
"_index": "customers",
"_id": "username3"
}
}
},
{
"_index": "customers",
"_id": "LrJGYowBtsOVXdO2r74I",
"_score": 1.0,
"_source": {
"username": "username3",
"first_name": "Lezlie",
"last_name": "Dunbabin",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1978-02-28",
"address": {
"street": "4 Westerfield Circle",
"city": "Orlando",
"province": "Florida",
"country": "United States",
"zip_code": "32825"
},
"hobbies": [
"Soccer",
"Reading"
],
"banks": [
{
"name": "BSI",
"account_number": 8176225
},
{
"name": "BRI",
"account_number": 9600877
},
{
"name": "BSI",
"account_number": 4487739
}
]
}
},
{
"_index": "customers",
"_id": "L7JGYowBtsOVXdO2r74I",
"_score": 1.0,
"_source": {
"index": {
"_index": "customers",
"_id": "username4"
}
}
},
{
"_index": "customers",
"_id": "MLJGYowBtsOVXdO2r74I",
"_score": 1.0,
"_source": {
"username": "username4",
"first_name": "Jamal",
"last_name": "Habard",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1977-10-29",
"address": {
"street": "01 Toban Place",
"city": "Schenectady",
"province": "New York",
"country": "United States",
"zip_code": "12305"
},
"hobbies": [
"Gaming",
"Soccer"
],
"banks": [
{
"name": "BCA Digital",
"account_number": 4429076
},
{
"name": "BCA",
"account_number": 6297767
}
]
}
},
{
"_index": "customers",
"_id": "MbJGYowBtsOVXdO2r74I",
"_score": 1.0,
"_source": {
"index": {
"_index": "customers",
"_id": "username5"
}
}
},
{
"_index": "customers",
"_id": "MrJGYowBtsOVXdO2r74I",
"_score": 1.0,
"_source": {
"username": "username5",
"first_name": "Broddy",
"last_name": "Speere",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1980-10-26",
"address": {
"street": "0586 Michigan Drive",
"city": "Saint Petersburg",
"province": "Florida",
"country": "United States",
"zip_code": "33715"
},
"hobbies": [
"Gaming",
"Reading"
],
"banks": [
{
"name": "Mandiri",
"account_number": 1852753
}
]
}
}
]
}
}Paging dan Sorting
You need to implement paging and sorting when using the Search API, but using Query Parameters can be complex and hard to read? You can simplify this process by using the Request Body for paging and sorting. It’s a more straightforward and readable approach than using Query Parameters.
# search using match_all with paging and sorting
# POST http://localhost:9200/customers/_search
response = es.search(index='customers', body={
'query': {
'match_all': {}
},
'size': 10,
'from': 0,
'sort': [
{
"username.keyword": {
"order": "desc"
}
}
]
})
print(json.dumps(response.body, indent=4)){
"took": 4,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2000,
"relation": "eq"
},
"max_score": null,
"hits": [
{
"_index": "customers",
"_id": "9rJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username999",
"first_name": "Hali",
"last_name": "Worboy",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1988-06-02",
"address": {
"street": "2988 Delladonna Street",
"city": "Tallahassee",
"province": "Florida",
"country": "United States",
"zip_code": "32304"
},
"hobbies": [
"Basket",
"Gaming"
],
"banks": [
{
"name": "BTN",
"account_number": 2125724
},
{
"name": "BCA Digital",
"account_number": 9143118
},
{
"name": "BTN",
"account_number": 2713733
}
]
},
"sort": [
"username999"
]
},
{
"_index": "customers",
"_id": "9LJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username998",
"first_name": "Bogey",
"last_name": "Muldowney",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1977-07-23",
"address": {
"street": "25 Eastwood Pass",
"city": "Washington",
"province": "District of Columbia",
"country": "United States",
"zip_code": "20210"
},
"hobbies": [
"Coding",
"Soccer"
],
"banks": [
{
"name": "BTN",
"account_number": 5412752
},
{
"name": "BNI",
"account_number": 4775113
}
]
},
"sort": [
"username998"
]
},
{
"_index": "customers",
"_id": "8rJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username997",
"first_name": "Christabella",
"last_name": "Quincey",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1984-02-27",
"address": {
"street": "5697 Arapahoe Trail",
"city": "Washington",
"province": "District of Columbia",
"country": "United States",
"zip_code": "20566"
},
"hobbies": [
"Traveling",
"Basket"
],
"banks": [
{
"name": "BCA",
"account_number": 4700078
},
{
"name": "BSI",
"account_number": 8762184
},
{
"name": "Jago",
"account_number": 8717479
}
]
},
"sort": [
"username997"
]
},
{
"_index": "customers",
"_id": "8LJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username996",
"first_name": "Conrade",
"last_name": "Scardifeild",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1971-01-29",
"address": {
"street": "691 Bartillon Point",
"city": "Chesapeake",
"province": "Virginia",
"country": "United States",
"zip_code": "23324"
},
"hobbies": [
"Traveling",
"Gaming"
],
"banks": [
{
"name": "BRI",
"account_number": 6489073
},
{
"name": "Jago",
"account_number": 1347358
},
{
"name": "Mandiri",
"account_number": 9614049
}
]
},
"sort": [
"username996"
]
},
{
"_index": "customers",
"_id": "7rJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username995",
"first_name": "Kelley",
"last_name": "Nuccitelli",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1986-05-15",
"address": {
"street": "7 Glacier Hill Terrace",
"city": "Oakland",
"province": "California",
"country": "United States",
"zip_code": "94611"
},
"hobbies": [
"Traveling",
"Soccer"
],
"banks": [
{
"name": "Jago",
"account_number": 3103617
}
]
},
"sort": [
"username995"
]
},
{
"_index": "customers",
"_id": "7LJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username994",
"first_name": "Ardra",
"last_name": "Petkov",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1981-09-25",
"address": {
"street": "35248 Emmet Plaza",
"city": "Rochester",
"province": "New York",
"country": "United States",
"zip_code": "14683"
},
"hobbies": [
"Gaming",
"Coding"
],
"banks": [
{
"name": "BCA",
"account_number": 1837241
}
]
},
"sort": [
"username994"
]
},
{
"_index": "customers",
"_id": "6rJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username993",
"first_name": "Hendrick",
"last_name": "Kybert",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1987-09-30",
"address": {
"street": "8 Mayfield Circle",
"city": "Annapolis",
"province": "Maryland",
"country": "United States",
"zip_code": "21405"
},
"hobbies": [
"Traveling",
"Basket"
],
"banks": [
{
"name": "BSI",
"account_number": 4702117
}
]
},
"sort": [
"username993"
]
},
{
"_index": "customers",
"_id": "6LJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username992",
"first_name": "Donnie",
"last_name": "Fanshaw",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1989-01-14",
"address": {
"street": "88 Moulton Center",
"city": "Denton",
"province": "Texas",
"country": "United States",
"zip_code": "76205"
},
"hobbies": [
"Gaming",
"Traveling"
],
"banks": [
{
"name": "BTN",
"account_number": 3054684
},
{
"name": "BRI",
"account_number": 4976349
}
]
},
"sort": [
"username992"
]
},
{
"_index": "customers",
"_id": "5rJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username991",
"first_name": "Vivien",
"last_name": "Tetla",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1993-12-07",
"address": {
"street": "383 Sutherland Terrace",
"city": "Chicago",
"province": "Illinois",
"country": "United States",
"zip_code": "60630"
},
"hobbies": [
"Coding",
"Reading"
],
"banks": [
{
"name": "BNI",
"account_number": 2729641
},
{
"name": "BCA Digital",
"account_number": 2351409
}
]
},
"sort": [
"username991"
]
},
{
"_index": "customers",
"_id": "5LJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username990",
"first_name": "Curry",
"last_name": "Mahaddie",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1993-10-11",
"address": {
"street": "39280 West Park",
"city": "Racine",
"province": "Wisconsin",
"country": "United States",
"zip_code": "53405"
},
"hobbies": [
"Gaming",
"Soccer"
],
"banks": [
{
"name": "BSI",
"account_number": 4703593
},
{
"name": "Jago",
"account_number": 4309892
},
{
"name": "BCA Digital",
"account_number": 8581300
}
]
},
"sort": [
"username990"
]
}
]
}
}Term Query
You need an efficient way to search for exact values (like usernames, product IDs, prices, etc.) in your data. However, using the Term Query for text data types might lead to inaccurate results due to text analysis (like automatic lowercasing, symbol removal, etc.).
Instead of using Term Query for text data types, it’s more appropriate to use Match Query as it can handle the peculiarities of text analysis and provide more accurate results.
# select * from customers where gender = 'Female' limit 10 offset 0 order by username desc
# POST http://localhost:9200/customers/_search
response = es.search(index='customers', body={
'query': {
'term': {
'gender.keyword': 'Female'
}
},
'size': 10,
'from': 0,
'sort': [
{
"username.keyword": {
"order": "desc"
}
}
]
})
print(json.dumps(response.body, indent=4)){
"took": 4,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 484,
"relation": "eq"
},
"max_score": null,
"hits": [
{
"_index": "customers",
"_id": "9rJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username999",
"first_name": "Hali",
"last_name": "Worboy",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1988-06-02",
"address": {
"street": "2988 Delladonna Street",
"city": "Tallahassee",
"province": "Florida",
"country": "United States",
"zip_code": "32304"
},
"hobbies": [
"Basket",
"Gaming"
],
"banks": [
{
"name": "BTN",
"account_number": 2125724
},
{
"name": "BCA Digital",
"account_number": 9143118
},
{
"name": "BTN",
"account_number": 2713733
}
]
},
"sort": [
"username999"
]
},
{
"_index": "customers",
"_id": "8rJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username997",
"first_name": "Christabella",
"last_name": "Quincey",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1984-02-27",
"address": {
"street": "5697 Arapahoe Trail",
"city": "Washington",
"province": "District of Columbia",
"country": "United States",
"zip_code": "20566"
},
"hobbies": [
"Traveling",
"Basket"
],
"banks": [
{
"name": "BCA",
"account_number": 4700078
},
{
"name": "BSI",
"account_number": 8762184
},
{
"name": "Jago",
"account_number": 8717479
}
]
},
"sort": [
"username997"
]
},
{
"_index": "customers",
"_id": "7LJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username994",
"first_name": "Ardra",
"last_name": "Petkov",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1981-09-25",
"address": {
"street": "35248 Emmet Plaza",
"city": "Rochester",
"province": "New York",
"country": "United States",
"zip_code": "14683"
},
"hobbies": [
"Gaming",
"Coding"
],
"banks": [
{
"name": "BCA",
"account_number": 1837241
}
]
},
"sort": [
"username994"
]
},
{
"_index": "customers",
"_id": "5rJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username991",
"first_name": "Vivien",
"last_name": "Tetla",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1993-12-07",
"address": {
"street": "383 Sutherland Terrace",
"city": "Chicago",
"province": "Illinois",
"country": "United States",
"zip_code": "60630"
},
"hobbies": [
"Coding",
"Reading"
],
"banks": [
{
"name": "BNI",
"account_number": 2729641
},
{
"name": "BCA Digital",
"account_number": 2351409
}
]
},
"sort": [
"username991"
]
},
{
"_index": "customers",
"_id": "3rJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username987",
"first_name": "Binni",
"last_name": "Grason",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1995-03-25",
"address": {
"street": "8237 Merry Street",
"city": "Ogden",
"province": "Utah",
"country": "United States",
"zip_code": "84409"
},
"hobbies": [
"Coding",
"Soccer"
],
"banks": [
{
"name": "BRI",
"account_number": 9766778
},
{
"name": "BTPN",
"account_number": 4851415
},
{
"name": "BRI",
"account_number": 8064204
}
]
},
"sort": [
"username987"
]
},
{
"_index": "customers",
"_id": "3LJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username986",
"first_name": "Eden",
"last_name": "Carress",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1976-04-15",
"address": {
"street": "30 Washington Point",
"city": "Knoxville",
"province": "Tennessee",
"country": "United States",
"zip_code": "37914"
},
"hobbies": [
"Reading",
"Traveling"
],
"banks": [
{
"name": "Jago",
"account_number": 1649494
},
{
"name": "BTPN",
"account_number": 8919650
},
{
"name": "BCA Digital",
"account_number": 8638882
}
]
},
"sort": [
"username986"
]
},
{
"_index": "customers",
"_id": "2rJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username985",
"first_name": "Barb",
"last_name": "Cowap",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1998-07-04",
"address": {
"street": "90566 Troy Road",
"city": "Jackson",
"province": "Mississippi",
"country": "United States",
"zip_code": "39236"
},
"hobbies": [
"Coding",
"Traveling"
],
"banks": [
{
"name": "BTPN",
"account_number": 8044372
},
{
"name": "BTN",
"account_number": 8755118
},
{
"name": "BCA",
"account_number": 3735619
}
]
},
"sort": [
"username985"
]
},
{
"_index": "customers",
"_id": "1rJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username983",
"first_name": "Reena",
"last_name": "Stitle",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1981-10-08",
"address": {
"street": "3 Nelson Court",
"city": "Norfolk",
"province": "Virginia",
"country": "United States",
"zip_code": "23504"
},
"hobbies": [
"Soccer",
"Traveling"
],
"banks": [
{
"name": "BTN",
"account_number": 8018980
},
{
"name": "Mandiri",
"account_number": 2091947
}
]
},
"sort": [
"username983"
]
},
{
"_index": "customers",
"_id": "0rJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username981",
"first_name": "Dido",
"last_name": "Peet",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1995-02-15",
"address": {
"street": "963 Oneill Plaza",
"city": "Sacramento",
"province": "California",
"country": "United States",
"zip_code": "95833"
},
"hobbies": [
"Basket",
"Gaming"
],
"banks": [
{
"name": "BSI",
"account_number": 9681008
},
{
"name": "BTN",
"account_number": 6144820
},
{
"name": "Mandiri",
"account_number": 3259997
}
]
},
"sort": [
"username981"
]
},
{
"_index": "customers",
"_id": "0LJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username980",
"first_name": "Gusti",
"last_name": "Bickerdike",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1988-10-25",
"address": {
"street": "60203 Hooker Avenue",
"city": "New York City",
"province": "New York",
"country": "United States",
"zip_code": "10170"
},
"hobbies": [
"Traveling",
"Soccer"
],
"banks": [
{
"name": "BTPN",
"account_number": 6856311
},
{
"name": "BNI",
"account_number": 3246052
},
{
"name": "Mandiri",
"account_number": 6924209
}
]
},
"sort": [
"username980"
]
}
]
}
}# select * from customers where banks.name = 'BCA' limit 10 offset 0 order by username desc
# POST http://localhost:9200/customers/_search
response = es.search(index='customers', body={
'query': {
'term': {
'banks.name.keyword': 'BCA'
}
},
'size': 10,
'from': 0,
'sort': [
{
"username.keyword": {
"order": "desc"
}
}
]
})
print(json.dumps(response.body, indent=4)){
"took": 4,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 192,
"relation": "eq"
},
"max_score": null,
"hits": [
{
"_index": "customers",
"_id": "8rJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username997",
"first_name": "Christabella",
"last_name": "Quincey",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1984-02-27",
"address": {
"street": "5697 Arapahoe Trail",
"city": "Washington",
"province": "District of Columbia",
"country": "United States",
"zip_code": "20566"
},
"hobbies": [
"Traveling",
"Basket"
],
"banks": [
{
"name": "BCA",
"account_number": 4700078
},
{
"name": "BSI",
"account_number": 8762184
},
{
"name": "Jago",
"account_number": 8717479
}
]
},
"sort": [
"username997"
]
},
{
"_index": "customers",
"_id": "7LJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username994",
"first_name": "Ardra",
"last_name": "Petkov",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1981-09-25",
"address": {
"street": "35248 Emmet Plaza",
"city": "Rochester",
"province": "New York",
"country": "United States",
"zip_code": "14683"
},
"hobbies": [
"Gaming",
"Coding"
],
"banks": [
{
"name": "BCA",
"account_number": 1837241
}
]
},
"sort": [
"username994"
]
},
{
"_index": "customers",
"_id": "2rJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username985",
"first_name": "Barb",
"last_name": "Cowap",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1998-07-04",
"address": {
"street": "90566 Troy Road",
"city": "Jackson",
"province": "Mississippi",
"country": "United States",
"zip_code": "39236"
},
"hobbies": [
"Coding",
"Traveling"
],
"banks": [
{
"name": "BTPN",
"account_number": 8044372
},
{
"name": "BTN",
"account_number": 8755118
},
{
"name": "BCA",
"account_number": 3735619
}
]
},
"sort": [
"username985"
]
},
{
"_index": "customers",
"_id": "zLJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username978",
"first_name": "Cassondra",
"last_name": "Leppingwell",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1987-10-17",
"address": {
"street": "01 Almo Way",
"city": "Kansas City",
"province": "Missouri",
"country": "United States",
"zip_code": "64125"
},
"hobbies": [
"Coding",
"Traveling"
],
"banks": [
{
"name": "BCA",
"account_number": 8796779
},
{
"name": "BSI",
"account_number": 4952291
},
{
"name": "BRI",
"account_number": 8057040
}
]
},
"sort": [
"username978"
]
},
{
"_index": "customers",
"_id": "vLJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username970",
"first_name": "Terrye",
"last_name": "Greatbanks",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1971-07-18",
"address": {
"street": "43271 Sunfield Park",
"city": "San Francisco",
"province": "California",
"country": "United States",
"zip_code": "94121"
},
"hobbies": [
"Reading",
"Coding"
],
"banks": [
{
"name": "BRI",
"account_number": 8085861
},
{
"name": "BRI",
"account_number": 4514681
},
{
"name": "BCA",
"account_number": 5571789
}
]
},
"sort": [
"username970"
]
},
{
"_index": "customers",
"_id": "urJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username969",
"first_name": "Lindi",
"last_name": "Odby",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1971-12-21",
"address": {
"street": "4 Summer Ridge Hill",
"city": "Bethlehem",
"province": "Pennsylvania",
"country": "United States",
"zip_code": "18018"
},
"hobbies": [
"Traveling",
"Soccer"
],
"banks": [
{
"name": "Jago",
"account_number": 3986142
},
{
"name": "BTPN",
"account_number": 3158564
},
{
"name": "BCA",
"account_number": 5127059
}
]
},
"sort": [
"username969"
]
},
{
"_index": "customers",
"_id": "prJGYowBtsOVXdO2sMWg",
"_score": null,
"_source": {
"username": "username959",
"first_name": "Buck",
"last_name": "Winks",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1994-04-02",
"address": {
"street": "314 Summit Alley",
"city": "Rochester",
"province": "New York",
"country": "United States",
"zip_code": "14609"
},
"hobbies": [
"Traveling",
"Gaming"
],
"banks": [
{
"name": "Jago",
"account_number": 2787784
},
{
"name": "Jago",
"account_number": 3217420
},
{
"name": "BCA",
"account_number": 5250594
}
]
},
"sort": [
"username959"
]
},
{
"_index": "customers",
"_id": "nLJGYowBtsOVXdO2sMWg",
"_score": null,
"_source": {
"username": "username954",
"first_name": "Annelise",
"last_name": "Carlesi",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1976-01-27",
"address": {
"street": "773 Luster Center",
"city": "Washington",
"province": "District of Columbia",
"country": "United States",
"zip_code": "20470"
},
"hobbies": [
"Gaming",
"Reading"
],
"banks": [
{
"name": "BCA",
"account_number": 3408116
}
]
},
"sort": [
"username954"
]
},
{
"_index": "customers",
"_id": "mrJGYowBtsOVXdO2sMWg",
"_score": null,
"_source": {
"username": "username953",
"first_name": "Axel",
"last_name": "Durning",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1998-03-15",
"address": {
"street": "9723 Superior Pass",
"city": "Knoxville",
"province": "Tennessee",
"country": "United States",
"zip_code": "37939"
},
"hobbies": [
"Coding",
"Soccer"
],
"banks": [
{
"name": "BCA",
"account_number": 8238872
},
{
"name": "BSI",
"account_number": 9331017
}
]
},
"sort": [
"username953"
]
},
{
"_index": "customers",
"_id": "lLJGYowBtsOVXdO2sMWg",
"_score": null,
"_source": {
"username": "username950",
"first_name": "Phebe",
"last_name": "McIlwrath",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1995-04-06",
"address": {
"street": "8 Mosinee Crossing",
"city": "Oakland",
"province": "California",
"country": "United States",
"zip_code": "94627"
},
"hobbies": [
"Basket",
"Traveling"
],
"banks": [
{
"name": "BCA",
"account_number": 2677894
},
{
"name": "BSI",
"account_number": 6574332
},
{
"name": "BRI",
"account_number": 5697471
}
]
},
"sort": [
"username950"
]
}
]
}
}Match Query
You’re working with text data types, and Term Query doesn’t provide accurate results because it doesn’t use the same text analysis methods.
Use Match Query, which, unlike Term Query, incorporates the same text analysis as the attribute being searched. Therefore, it’s suitable for text data types and can automatically apply the correct text analysis. Though, it’s worth noting that Match Query can be used for all data types. The topic of text analysis is quite complex and will be covered in a separate module.
# select * from customers where banks.name = 'BCA' limit 10 offset 0 order by username desc
# POST http://localhost:9200/customers/_search
response = es.search(index='customers', body={
'query': {
'match': {
'banks.name.keyword': 'BCA'
}
},
'size': 10,
'from': 0,
'sort': [
{
"username.keyword": {
"order": "desc"
}
}
]
})
print(json.dumps(response.body, indent=4)){
"took": 3,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 192,
"relation": "eq"
},
"max_score": null,
"hits": [
{
"_index": "customers",
"_id": "8rJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username997",
"first_name": "Christabella",
"last_name": "Quincey",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1984-02-27",
"address": {
"street": "5697 Arapahoe Trail",
"city": "Washington",
"province": "District of Columbia",
"country": "United States",
"zip_code": "20566"
},
"hobbies": [
"Traveling",
"Basket"
],
"banks": [
{
"name": "BCA",
"account_number": 4700078
},
{
"name": "BSI",
"account_number": 8762184
},
{
"name": "Jago",
"account_number": 8717479
}
]
},
"sort": [
"username997"
]
},
{
"_index": "customers",
"_id": "7LJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username994",
"first_name": "Ardra",
"last_name": "Petkov",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1981-09-25",
"address": {
"street": "35248 Emmet Plaza",
"city": "Rochester",
"province": "New York",
"country": "United States",
"zip_code": "14683"
},
"hobbies": [
"Gaming",
"Coding"
],
"banks": [
{
"name": "BCA",
"account_number": 1837241
}
]
},
"sort": [
"username994"
]
},
{
"_index": "customers",
"_id": "2rJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username985",
"first_name": "Barb",
"last_name": "Cowap",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1998-07-04",
"address": {
"street": "90566 Troy Road",
"city": "Jackson",
"province": "Mississippi",
"country": "United States",
"zip_code": "39236"
},
"hobbies": [
"Coding",
"Traveling"
],
"banks": [
{
"name": "BTPN",
"account_number": 8044372
},
{
"name": "BTN",
"account_number": 8755118
},
{
"name": "BCA",
"account_number": 3735619
}
]
},
"sort": [
"username985"
]
},
{
"_index": "customers",
"_id": "zLJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username978",
"first_name": "Cassondra",
"last_name": "Leppingwell",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1987-10-17",
"address": {
"street": "01 Almo Way",
"city": "Kansas City",
"province": "Missouri",
"country": "United States",
"zip_code": "64125"
},
"hobbies": [
"Coding",
"Traveling"
],
"banks": [
{
"name": "BCA",
"account_number": 8796779
},
{
"name": "BSI",
"account_number": 4952291
},
{
"name": "BRI",
"account_number": 8057040
}
]
},
"sort": [
"username978"
]
},
{
"_index": "customers",
"_id": "vLJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username970",
"first_name": "Terrye",
"last_name": "Greatbanks",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1971-07-18",
"address": {
"street": "43271 Sunfield Park",
"city": "San Francisco",
"province": "California",
"country": "United States",
"zip_code": "94121"
},
"hobbies": [
"Reading",
"Coding"
],
"banks": [
{
"name": "BRI",
"account_number": 8085861
},
{
"name": "BRI",
"account_number": 4514681
},
{
"name": "BCA",
"account_number": 5571789
}
]
},
"sort": [
"username970"
]
},
{
"_index": "customers",
"_id": "urJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username969",
"first_name": "Lindi",
"last_name": "Odby",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1971-12-21",
"address": {
"street": "4 Summer Ridge Hill",
"city": "Bethlehem",
"province": "Pennsylvania",
"country": "United States",
"zip_code": "18018"
},
"hobbies": [
"Traveling",
"Soccer"
],
"banks": [
{
"name": "Jago",
"account_number": 3986142
},
{
"name": "BTPN",
"account_number": 3158564
},
{
"name": "BCA",
"account_number": 5127059
}
]
},
"sort": [
"username969"
]
},
{
"_index": "customers",
"_id": "prJGYowBtsOVXdO2sMWg",
"_score": null,
"_source": {
"username": "username959",
"first_name": "Buck",
"last_name": "Winks",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1994-04-02",
"address": {
"street": "314 Summit Alley",
"city": "Rochester",
"province": "New York",
"country": "United States",
"zip_code": "14609"
},
"hobbies": [
"Traveling",
"Gaming"
],
"banks": [
{
"name": "Jago",
"account_number": 2787784
},
{
"name": "Jago",
"account_number": 3217420
},
{
"name": "BCA",
"account_number": 5250594
}
]
},
"sort": [
"username959"
]
},
{
"_index": "customers",
"_id": "nLJGYowBtsOVXdO2sMWg",
"_score": null,
"_source": {
"username": "username954",
"first_name": "Annelise",
"last_name": "Carlesi",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1976-01-27",
"address": {
"street": "773 Luster Center",
"city": "Washington",
"province": "District of Columbia",
"country": "United States",
"zip_code": "20470"
},
"hobbies": [
"Gaming",
"Reading"
],
"banks": [
{
"name": "BCA",
"account_number": 3408116
}
]
},
"sort": [
"username954"
]
},
{
"_index": "customers",
"_id": "mrJGYowBtsOVXdO2sMWg",
"_score": null,
"_source": {
"username": "username953",
"first_name": "Axel",
"last_name": "Durning",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1998-03-15",
"address": {
"street": "9723 Superior Pass",
"city": "Knoxville",
"province": "Tennessee",
"country": "United States",
"zip_code": "37939"
},
"hobbies": [
"Coding",
"Soccer"
],
"banks": [
{
"name": "BCA",
"account_number": 8238872
},
{
"name": "BSI",
"account_number": 9331017
}
]
},
"sort": [
"username953"
]
},
{
"_index": "customers",
"_id": "lLJGYowBtsOVXdO2sMWg",
"_score": null,
"_source": {
"username": "username950",
"first_name": "Phebe",
"last_name": "McIlwrath",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1995-04-06",
"address": {
"street": "8 Mosinee Crossing",
"city": "Oakland",
"province": "California",
"country": "United States",
"zip_code": "94627"
},
"hobbies": [
"Basket",
"Traveling"
],
"banks": [
{
"name": "BCA",
"account_number": 2677894
},
{
"name": "BSI",
"account_number": 6574332
},
{
"name": "BRI",
"account_number": 5697471
}
]
},
"sort": [
"username950"
]
}
]
}
}Standard Analyzer
For text data types, by default, the StandardAnalyzer is used. Simplistically, this is an object used to convert the original value into a searchable form of smaller tokens.
Here is the official documentation: StandardAnalyzer
- Example: When you use the text value
[email protected], theStandardAnalyzerchanges it intoaditira example com. This automatic process allows you to search using the wordsaditira,example, orcom.
Match Query Operator
When using Match Query, there’s an attribute called operator that can take the values OR or AND. By default, the operator value is OR.
Example: Therefore, when we search for
bca digital, theStandardAnalyzerwill break it down intobcaORdigital. As a result, all banks with ‘BCA’ or ‘BCA Digital’ will appear in the search results.Adjustment: If you want to search for records that contain both
bcaANDdigital, you need to change the query operator toAND.
# search customers
# POST http://localhost:9200/customers/_search
response = es.search(index='customers', body={
'query': {
'match': {
'banks.name': {
'query': 'bca digital',
'operator': 'AND'
}
}
},
'size': 10,
'from': 0,
'sort': [
{
"username.keyword": {
"order": "desc"
}
}
]
})
print(json.dumps(response.body, indent=4)){
"took": 3,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 217,
"relation": "eq"
},
"max_score": null,
"hits": [
{
"_index": "customers",
"_id": "9rJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username999",
"first_name": "Hali",
"last_name": "Worboy",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1988-06-02",
"address": {
"street": "2988 Delladonna Street",
"city": "Tallahassee",
"province": "Florida",
"country": "United States",
"zip_code": "32304"
},
"hobbies": [
"Basket",
"Gaming"
],
"banks": [
{
"name": "BTN",
"account_number": 2125724
},
{
"name": "BCA Digital",
"account_number": 9143118
},
{
"name": "BTN",
"account_number": 2713733
}
]
},
"sort": [
"username999"
]
},
{
"_index": "customers",
"_id": "5rJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username991",
"first_name": "Vivien",
"last_name": "Tetla",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1993-12-07",
"address": {
"street": "383 Sutherland Terrace",
"city": "Chicago",
"province": "Illinois",
"country": "United States",
"zip_code": "60630"
},
"hobbies": [
"Coding",
"Reading"
],
"banks": [
{
"name": "BNI",
"account_number": 2729641
},
{
"name": "BCA Digital",
"account_number": 2351409
}
]
},
"sort": [
"username991"
]
},
{
"_index": "customers",
"_id": "5LJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username990",
"first_name": "Curry",
"last_name": "Mahaddie",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1993-10-11",
"address": {
"street": "39280 West Park",
"city": "Racine",
"province": "Wisconsin",
"country": "United States",
"zip_code": "53405"
},
"hobbies": [
"Gaming",
"Soccer"
],
"banks": [
{
"name": "BSI",
"account_number": 4703593
},
{
"name": "Jago",
"account_number": 4309892
},
{
"name": "BCA Digital",
"account_number": 8581300
}
]
},
"sort": [
"username990"
]
},
{
"_index": "customers",
"_id": "7rJGYowBtsOVXdO2r74K",
"_score": null,
"_source": {
"username": "username99",
"first_name": "Merwyn",
"last_name": "Renzullo",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1997-06-19",
"address": {
"street": "05 Brickson Park Street",
"city": "Denton",
"province": "Texas",
"country": "United States",
"zip_code": "76205"
},
"hobbies": [
"Reading",
"Coding"
],
"banks": [
{
"name": "BCA Digital",
"account_number": 6770052
},
{
"name": "BSI",
"account_number": 9745839
},
{
"name": "Mandiri",
"account_number": 4358454
}
]
},
"sort": [
"username99"
]
},
{
"_index": "customers",
"_id": "3LJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username986",
"first_name": "Eden",
"last_name": "Carress",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1976-04-15",
"address": {
"street": "30 Washington Point",
"city": "Knoxville",
"province": "Tennessee",
"country": "United States",
"zip_code": "37914"
},
"hobbies": [
"Reading",
"Traveling"
],
"banks": [
{
"name": "Jago",
"account_number": 1649494
},
{
"name": "BTPN",
"account_number": 8919650
},
{
"name": "BCA Digital",
"account_number": 8638882
}
]
},
"sort": [
"username986"
]
},
{
"_index": "customers",
"_id": "1LJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username982",
"first_name": "Otto",
"last_name": "Kiff",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1990-12-16",
"address": {
"street": "222 Leroy Avenue",
"city": "Honolulu",
"province": "Hawaii",
"country": "United States",
"zip_code": "96835"
},
"hobbies": [
"Traveling",
"Reading"
],
"banks": [
{
"name": "BTN",
"account_number": 4599163
},
{
"name": "BNI",
"account_number": 3228246
},
{
"name": "BCA Digital",
"account_number": 6082404
}
]
},
"sort": [
"username982"
]
},
{
"_index": "customers",
"_id": "7LJGYowBtsOVXdO2r74K",
"_score": null,
"_source": {
"username": "username98",
"first_name": "Alyss",
"last_name": "Habbeshaw",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1978-10-06",
"address": {
"street": "832 Redwing Trail",
"city": "Young America",
"province": "Minnesota",
"country": "United States",
"zip_code": "55573"
},
"hobbies": [
"Traveling",
"Coding"
],
"banks": [
{
"name": "BCA Digital",
"account_number": 6809887
},
{
"name": "BCA Digital",
"account_number": 3732237
},
{
"name": "BRI",
"account_number": 6990718
}
]
},
"sort": [
"username98"
]
},
{
"_index": "customers",
"_id": "zrJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username979",
"first_name": "Zacharia",
"last_name": "O'Harney",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1987-08-03",
"address": {
"street": "52 Reindahl Way",
"city": "Tallahassee",
"province": "Florida",
"country": "United States",
"zip_code": "32304"
},
"hobbies": [
"Basket",
"Reading"
],
"banks": [
{
"name": "Jago",
"account_number": 5917638
},
{
"name": "BCA Digital",
"account_number": 1339521
},
{
"name": "BTPN",
"account_number": 7204366
}
]
},
"sort": [
"username979"
]
},
{
"_index": "customers",
"_id": "xLJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username974",
"first_name": "Thekla",
"last_name": "Zamora",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1973-01-03",
"address": {
"street": "2 Sherman Road",
"city": "Bethlehem",
"province": "Pennsylvania",
"country": "United States",
"zip_code": "18018"
},
"hobbies": [
"Gaming",
"Soccer"
],
"banks": [
{
"name": "BCA Digital",
"account_number": 5390077
},
{
"name": "BTPN",
"account_number": 8209397
},
{
"name": "Jago",
"account_number": 3715524
}
]
},
"sort": [
"username974"
]
},
{
"_index": "customers",
"_id": "rrJGYowBtsOVXdO2sMWi",
"_score": null,
"_source": {
"username": "username963",
"first_name": "Wheeler",
"last_name": "Bullene",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1993-09-25",
"address": {
"street": "2 Welch Street",
"city": "Gary",
"province": "Indiana",
"country": "United States",
"zip_code": "46406"
},
"hobbies": [
"Reading",
"Gaming"
],
"banks": [
{
"name": "BSI",
"account_number": 5441930
},
{
"name": "BCA Digital",
"account_number": 3117133
},
{
"name": "BTN",
"account_number": 9135063
}
]
},
"sort": [
"username963"
]
}
]
}
}Terms Query
- Elasticsearch includes a query that functions similarly to the
INoperator in SQL, which is theTerms Query. - However, remember that Terms Query behaves much like
Term Query, so it will not use the default Text Analysis of its attributes. - So, just like
Term Query,Terms Queryis also best suited for attributes that are not of Text data type.
# search customers
# POST http://localhost:9200/customers/_search
response = es.search(index='customers', body={
'query': {
'terms': {
'username': [
'username1',
'username2',
'username3',
]
}
},
})
print(json.dumps(response.body, indent=4)){
"took": 2,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 3,
"relation": "eq"
},
"max_score": 1.0,
"hits": [
{
"_index": "customers",
"_id": "KrJGYowBtsOVXdO2r74I",
"_score": 1.0,
"_source": {
"username": "username1",
"first_name": "Rollie",
"last_name": "Farge",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1984-11-23",
"address": {
"street": "227 Eastwood Pass",
"city": "New York City",
"province": "New York",
"country": "United States",
"zip_code": "10131"
},
"hobbies": [
"Coding",
"Gaming"
],
"banks": [
{
"name": "Mandiri",
"account_number": 8949575
},
{
"name": "Mandiri",
"account_number": 9256376
},
{
"name": "Mandiri",
"account_number": 7904606
}
]
}
},
{
"_index": "customers",
"_id": "LLJGYowBtsOVXdO2r74I",
"_score": 1.0,
"_source": {
"username": "username2",
"first_name": "Toinette",
"last_name": "Ketteridge",
"email": "[email protected]",
"gender": "Female",
"birth_date": "2000-06-07",
"address": {
"street": "48 Golf View Point",
"city": "Youngstown",
"province": "Ohio",
"country": "United States",
"zip_code": "44505"
},
"hobbies": [
"Reading",
"Coding"
],
"banks": [
{
"name": "BNI",
"account_number": 7051376
},
{
"name": "BNI",
"account_number": 9284273
}
]
}
},
{
"_index": "customers",
"_id": "LrJGYowBtsOVXdO2r74I",
"_score": 1.0,
"_source": {
"username": "username3",
"first_name": "Lezlie",
"last_name": "Dunbabin",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1978-02-28",
"address": {
"street": "4 Westerfield Circle",
"city": "Orlando",
"province": "Florida",
"country": "United States",
"zip_code": "32825"
},
"hobbies": [
"Soccer",
"Reading"
],
"banks": [
{
"name": "BSI",
"account_number": 8176225
},
{
"name": "BRI",
"account_number": 9600877
},
{
"name": "BSI",
"account_number": 4487739
}
]
}
}
]
}
}Boolean Query
What if we want to search multiple fields? Term and Match are used to search a single field. If we want to search multiple fields at once, we can combine them using a Boolean Query.
Boolean Query Field
| Occur | Description |
|---|---|
| must | The query has to appear in the resulting document and contribute to the score |
| filter | The query must appear in the resulting document, but does not contribute to the score |
| must_not | The query should not appear in the resulting document |
| should | The query may appear (but is not required) in the resulting document |
Boolean Query - Must
# search customers
# POST http://localhost:9200/customers/_search
response = es.search(index='customers', body={
'query': {
'bool': {
'must': [
{
'term': {
'hobbies': 'gaming'
}
},
{
'term': {
'banks.name': 'bca'
}
}
]
}
},
})
print(json.dumps(response.body, indent=4)){
"took": 4,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 106,
"relation": "eq"
},
"max_score": 2.405724,
"hits": [
{
"_index": "customers",
"_id": "grJGYowBtsOVXdO2r74I",
"_score": 2.405724,
"_source": {
"username": "username45",
"first_name": "Durante",
"last_name": "Plampin",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1978-09-21",
"address": {
"street": "2132 Homewood Avenue",
"city": "New York City",
"province": "New York",
"country": "United States",
"zip_code": "10270"
},
"hobbies": [
"Soccer",
"Gaming"
],
"banks": [
{
"name": "BCA",
"account_number": 7378718
}
]
}
},
{
"_index": "customers",
"_id": "srJGYowBtsOVXdO2r74I",
"_score": 2.405724,
"_source": {
"username": "username69",
"first_name": "Gustave",
"last_name": "Biermatowicz",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1981-04-16",
"address": {
"street": "3904 Rowland Circle",
"city": "Montgomery",
"province": "Alabama",
"country": "United States",
"zip_code": "36177"
},
"hobbies": [
"Gaming",
"Coding"
],
"banks": [
{
"name": "BCA",
"account_number": 1117171
}
]
}
},
{
"_index": "customers",
"_id": "KrJGYowBtsOVXdO2r78K",
"_score": 2.405724,
"_source": {
"username": "username129",
"first_name": "Loralyn",
"last_name": "Sellars",
"email": "[email protected]",
"gender": "Female",
"birth_date": "2000-12-01",
"address": {
"street": "118 Oneill Hill",
"city": "Boulder",
"province": "Colorado",
"country": "United States",
"zip_code": "80305"
},
"hobbies": [
"Reading",
"Gaming"
],
"banks": [
{
"name": "BCA",
"account_number": 9013774
}
]
}
},
{
"_index": "customers",
"_id": "xLJGYowBtsOVXdO2r78K",
"_score": 2.405724,
"_source": {
"username": "username206",
"first_name": "Rooney",
"last_name": "Macias",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1990-09-20",
"address": {
"street": "84 Bultman Terrace",
"city": "Honolulu",
"province": "Hawaii",
"country": "United States",
"zip_code": "96835"
},
"hobbies": [
"Gaming",
"Coding"
],
"banks": [
{
"name": "BCA",
"account_number": 8841860
}
]
}
},
{
"_index": "customers",
"_id": "RrJGYowBtsOVXdO2r8HA",
"_score": 2.405724,
"_source": {
"username": "username399",
"first_name": "Craggie",
"last_name": "Greenway",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1983-06-24",
"address": {
"street": "0 Victoria Plaza",
"city": "Detroit",
"province": "Michigan",
"country": "United States",
"zip_code": "48232"
},
"hobbies": [
"Traveling",
"Gaming"
],
"banks": [
{
"name": "BCA",
"account_number": 7468098
}
]
}
},
{
"_index": "customers",
"_id": "TrJGYowBtsOVXdO2sMIZ",
"_score": 2.405724,
"_source": {
"username": "username531",
"first_name": "Gaby",
"last_name": "Imbrey",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1972-04-14",
"address": {
"street": "67587 Goodland Plaza",
"city": "Dallas",
"province": "Texas",
"country": "United States",
"zip_code": "75372"
},
"hobbies": [
"Reading",
"Gaming"
],
"banks": [
{
"name": "BCA",
"account_number": 1531772
}
]
}
},
{
"_index": "customers",
"_id": "VrJGYowBtsOVXdO2sMIZ",
"_score": 2.405724,
"_source": {
"username": "username535",
"first_name": "Rickert",
"last_name": "Egel",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1981-06-26",
"address": {
"street": "7 Myrtle Pass",
"city": "Jackson",
"province": "Mississippi",
"country": "United States",
"zip_code": "39236"
},
"hobbies": [
"Soccer",
"Gaming"
],
"banks": [
{
"name": "BCA",
"account_number": 1771757
}
]
}
},
{
"_index": "customers",
"_id": "gLJGYowBtsOVXdO2sMSg",
"_score": 2.405724,
"_source": {
"username": "username812",
"first_name": "Carey",
"last_name": "Maudlen",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1970-11-02",
"address": {
"street": "96 Schlimgen Hill",
"city": "New Orleans",
"province": "Louisiana",
"country": "United States",
"zip_code": "70174"
},
"hobbies": [
"Basket",
"Gaming"
],
"banks": [
{
"name": "BCA",
"account_number": 4270060
}
]
}
},
{
"_index": "customers",
"_id": "qLJGYowBtsOVXdO2sMSg",
"_score": 2.405724,
"_source": {
"username": "username832",
"first_name": "Mateo",
"last_name": "Brodhead",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1982-10-26",
"address": {
"street": "6661 High Crossing Crossing",
"city": "Sacramento",
"province": "California",
"country": "United States",
"zip_code": "94291"
},
"hobbies": [
"Traveling",
"Gaming"
],
"banks": [
{
"name": "BCA",
"account_number": 2425017
}
]
}
},
{
"_index": "customers",
"_id": "7rJGYowBtsOVXdO2sMSg",
"_score": 2.405724,
"_source": {
"username": "username867",
"first_name": "Aura",
"last_name": "Janacek",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1974-04-28",
"address": {
"street": "97788 Wayridge Alley",
"city": "El Paso",
"province": "Texas",
"country": "United States",
"zip_code": "88519"
},
"hobbies": [
"Soccer",
"Gaming"
],
"banks": [
{
"name": "BCA",
"account_number": 7495508
}
]
}
}
]
}
}Boolean Query - Filter
# search customers
# POST http://localhost:9200/customers/_search
response = es.search(index='customers', body={
'query': {
'bool': {
'filter': [
{
'term': {
'hobbies': 'gaming'
}
},
{
'term': {
'banks.name': 'bca'
}
}
]
}
},
})
print(json.dumps(response.body, indent=4)){
"took": 3,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 106,
"relation": "eq"
},
"max_score": 0.0,
"hits": [
{
"_index": "customers",
"_id": "MLJGYowBtsOVXdO2r74I",
"_score": 0.0,
"_source": {
"username": "username4",
"first_name": "Jamal",
"last_name": "Habard",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1977-10-29",
"address": {
"street": "01 Toban Place",
"city": "Schenectady",
"province": "New York",
"country": "United States",
"zip_code": "12305"
},
"hobbies": [
"Gaming",
"Soccer"
],
"banks": [
{
"name": "BCA Digital",
"account_number": 4429076
},
{
"name": "BCA",
"account_number": 6297767
}
]
}
},
{
"_index": "customers",
"_id": "NrJGYowBtsOVXdO2r74I",
"_score": 0.0,
"_source": {
"username": "username7",
"first_name": "Pavia",
"last_name": "Thomasen",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1994-12-27",
"address": {
"street": "04 Stoughton Point",
"city": "Grand Rapids",
"province": "Michigan",
"country": "United States",
"zip_code": "49518"
},
"hobbies": [
"Soccer",
"Gaming"
],
"banks": [
{
"name": "BCA Digital",
"account_number": 3057759
},
{
"name": "Mandiri",
"account_number": 3500404
}
]
}
},
{
"_index": "customers",
"_id": "OLJGYowBtsOVXdO2r74I",
"_score": 0.0,
"_source": {
"username": "username8",
"first_name": "Sly",
"last_name": "O'Hagerty",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1984-04-09",
"address": {
"street": "58624 Anderson Plaza",
"city": "Saint Louis",
"province": "Missouri",
"country": "United States",
"zip_code": "63121"
},
"hobbies": [
"Reading",
"Gaming"
],
"banks": [
{
"name": "BCA Digital",
"account_number": 7399293
},
{
"name": "Jago",
"account_number": 1855103
}
]
}
},
{
"_index": "customers",
"_id": "WrJGYowBtsOVXdO2r74I",
"_score": 0.0,
"_source": {
"username": "username25",
"first_name": "Britteny",
"last_name": "Reading",
"email": "[email protected]",
"gender": "Female",
"birth_date": "2000-10-17",
"address": {
"street": "7 Sycamore Trail",
"city": "Kansas City",
"province": "Missouri",
"country": "United States",
"zip_code": "64130"
},
"hobbies": [
"Gaming",
"Traveling"
],
"banks": [
{
"name": "BSI",
"account_number": 7693299
},
{
"name": "BCA",
"account_number": 3416912
}
]
}
},
{
"_index": "customers",
"_id": "XrJGYowBtsOVXdO2r74I",
"_score": 0.0,
"_source": {
"username": "username27",
"first_name": "Arabele",
"last_name": "Critchley",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1986-05-05",
"address": {
"street": "9 Marquette Circle",
"city": "Huntsville",
"province": "Alabama",
"country": "United States",
"zip_code": "35805"
},
"hobbies": [
"Gaming",
"Soccer"
],
"banks": [
{
"name": "Mandiri",
"account_number": 7805389
},
{
"name": "BCA Digital",
"account_number": 3713601
},
{
"name": "BNI",
"account_number": 6752101
}
]
}
},
{
"_index": "customers",
"_id": "bLJGYowBtsOVXdO2r74I",
"_score": 0.0,
"_source": {
"username": "username34",
"first_name": "Salomo",
"last_name": "Stubbington",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1974-08-09",
"address": {
"street": "9297 Dorton Plaza",
"city": "El Paso",
"province": "Texas",
"country": "United States",
"zip_code": "79940"
},
"hobbies": [
"Coding",
"Gaming"
],
"banks": [
{
"name": "BRI",
"account_number": 6392777
},
{
"name": "BCA Digital",
"account_number": 8157580
}
]
}
},
{
"_index": "customers",
"_id": "cLJGYowBtsOVXdO2r74I",
"_score": 0.0,
"_source": {
"username": "username36",
"first_name": "Catlee",
"last_name": "Ivy",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1994-08-24",
"address": {
"street": "64347 7th Parkway",
"city": "New Orleans",
"province": "Louisiana",
"country": "United States",
"zip_code": "70149"
},
"hobbies": [
"Gaming",
"Soccer"
],
"banks": [
{
"name": "BCA",
"account_number": 5844642
},
{
"name": "BRI",
"account_number": 8309997
}
]
}
},
{
"_index": "customers",
"_id": "grJGYowBtsOVXdO2r74I",
"_score": 0.0,
"_source": {
"username": "username45",
"first_name": "Durante",
"last_name": "Plampin",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1978-09-21",
"address": {
"street": "2132 Homewood Avenue",
"city": "New York City",
"province": "New York",
"country": "United States",
"zip_code": "10270"
},
"hobbies": [
"Soccer",
"Gaming"
],
"banks": [
{
"name": "BCA",
"account_number": 7378718
}
]
}
},
{
"_index": "customers",
"_id": "sLJGYowBtsOVXdO2r74I",
"_score": 0.0,
"_source": {
"username": "username68",
"first_name": "Cathlene",
"last_name": "Hoyes",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1982-09-28",
"address": {
"street": "7 Schlimgen Center",
"city": "Kansas City",
"province": "Missouri",
"country": "United States",
"zip_code": "64125"
},
"hobbies": [
"Traveling",
"Gaming"
],
"banks": [
{
"name": "BCA",
"account_number": 7494105
},
{
"name": "BSI",
"account_number": 6897836
},
{
"name": "BCA Digital",
"account_number": 7987430
}
]
}
},
{
"_index": "customers",
"_id": "srJGYowBtsOVXdO2r74I",
"_score": 0.0,
"_source": {
"username": "username69",
"first_name": "Gustave",
"last_name": "Biermatowicz",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1981-04-16",
"address": {
"street": "3904 Rowland Circle",
"city": "Montgomery",
"province": "Alabama",
"country": "United States",
"zip_code": "36177"
},
"hobbies": [
"Gaming",
"Coding"
],
"banks": [
{
"name": "BCA",
"account_number": 1117171
}
]
}
}
]
}
}Boolean Query - Must Not
# search customers
# POST http://localhost:9200/customers/_search
response = es.search(index='customers', body={
'query': {
'bool': {
'must_not': [
{
'term': {
'hobbies': 'gaming'
}
},
{
'term': {
'banks.name': 'bca'
}
}
]
}
},
})
print(json.dumps(response.body, indent=4)){
"took": 3,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1412,
"relation": "eq"
},
"max_score": 0.0,
"hits": [
{
"_index": "customers",
"_id": "KbJGYowBtsOVXdO2r74I",
"_score": 0.0,
"_source": {
"index": {
"_index": "customers",
"_id": "username1"
}
}
},
{
"_index": "customers",
"_id": "K7JGYowBtsOVXdO2r74I",
"_score": 0.0,
"_source": {
"index": {
"_index": "customers",
"_id": "username2"
}
}
},
{
"_index": "customers",
"_id": "LLJGYowBtsOVXdO2r74I",
"_score": 0.0,
"_source": {
"username": "username2",
"first_name": "Toinette",
"last_name": "Ketteridge",
"email": "[email protected]",
"gender": "Female",
"birth_date": "2000-06-07",
"address": {
"street": "48 Golf View Point",
"city": "Youngstown",
"province": "Ohio",
"country": "United States",
"zip_code": "44505"
},
"hobbies": [
"Reading",
"Coding"
],
"banks": [
{
"name": "BNI",
"account_number": 7051376
},
{
"name": "BNI",
"account_number": 9284273
}
]
}
},
{
"_index": "customers",
"_id": "LbJGYowBtsOVXdO2r74I",
"_score": 0.0,
"_source": {
"index": {
"_index": "customers",
"_id": "username3"
}
}
},
{
"_index": "customers",
"_id": "LrJGYowBtsOVXdO2r74I",
"_score": 0.0,
"_source": {
"username": "username3",
"first_name": "Lezlie",
"last_name": "Dunbabin",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1978-02-28",
"address": {
"street": "4 Westerfield Circle",
"city": "Orlando",
"province": "Florida",
"country": "United States",
"zip_code": "32825"
},
"hobbies": [
"Soccer",
"Reading"
],
"banks": [
{
"name": "BSI",
"account_number": 8176225
},
{
"name": "BRI",
"account_number": 9600877
},
{
"name": "BSI",
"account_number": 4487739
}
]
}
},
{
"_index": "customers",
"_id": "L7JGYowBtsOVXdO2r74I",
"_score": 0.0,
"_source": {
"index": {
"_index": "customers",
"_id": "username4"
}
}
},
{
"_index": "customers",
"_id": "MbJGYowBtsOVXdO2r74I",
"_score": 0.0,
"_source": {
"index": {
"_index": "customers",
"_id": "username5"
}
}
},
{
"_index": "customers",
"_id": "M7JGYowBtsOVXdO2r74I",
"_score": 0.0,
"_source": {
"index": {
"_index": "customers",
"_id": "username6"
}
}
},
{
"_index": "customers",
"_id": "NbJGYowBtsOVXdO2r74I",
"_score": 0.0,
"_source": {
"index": {
"_index": "customers",
"_id": "username7"
}
}
},
{
"_index": "customers",
"_id": "N7JGYowBtsOVXdO2r74I",
"_score": 0.0,
"_source": {
"index": {
"_index": "customers",
"_id": "username8"
}
}
}
]
}
}Boolean Query - Should
# search customers
# POST http://localhost:9200/customers/_search
response = es.search(index='customers', body={
'query': {
'bool': {
'should': [
{
'term': {
'banks.name': 'bca'
}
},
{
'term': {
'banks.name': 'bri'
}
}
]
}
},
})
print(json.dumps(response.body, indent=4)){
"took": 5,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 538,
"relation": "eq"
},
"max_score": 2.6825786,
"hits": [
{
"_index": "customers",
"_id": "vLJGYowBtsOVXdO2sMWi",
"_score": 2.6825786,
"_source": {
"username": "username970",
"first_name": "Terrye",
"last_name": "Greatbanks",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1971-07-18",
"address": {
"street": "43271 Sunfield Park",
"city": "San Francisco",
"province": "California",
"country": "United States",
"zip_code": "94121"
},
"hobbies": [
"Reading",
"Coding"
],
"banks": [
{
"name": "BRI",
"account_number": 8085861
},
{
"name": "BRI",
"account_number": 4514681
},
{
"name": "BCA",
"account_number": 5571789
}
]
}
},
{
"_index": "customers",
"_id": "cLJGYowBtsOVXdO2r74I",
"_score": 2.5432863,
"_source": {
"username": "username36",
"first_name": "Catlee",
"last_name": "Ivy",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1994-08-24",
"address": {
"street": "64347 7th Parkway",
"city": "New Orleans",
"province": "Louisiana",
"country": "United States",
"zip_code": "70149"
},
"hobbies": [
"Gaming",
"Soccer"
],
"banks": [
{
"name": "BCA",
"account_number": 5844642
},
{
"name": "BRI",
"account_number": 8309997
}
]
}
},
{
"_index": "customers",
"_id": "CrJGYowBtsOVXdO2r8AK",
"_score": 2.5432863,
"_source": {
"username": "username241",
"first_name": "Warden",
"last_name": "Gaythwaite",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1982-02-15",
"address": {
"street": "596 Elmside Lane",
"city": "Naperville",
"province": "Illinois",
"country": "United States",
"zip_code": "60567"
},
"hobbies": [
"Coding",
"Reading"
],
"banks": [
{
"name": "BRI",
"account_number": 2665146
},
{
"name": "BCA",
"account_number": 9325424
}
]
}
},
{
"_index": "customers",
"_id": "trJGYowBtsOVXdO2r8HA",
"_score": 2.5432863,
"_source": {
"username": "username455",
"first_name": "Keir",
"last_name": "Haly",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1972-08-07",
"address": {
"street": "41 Dorton Point",
"city": "Milwaukee",
"province": "Wisconsin",
"country": "United States",
"zip_code": "53285"
},
"hobbies": [
"Reading",
"Soccer"
],
"banks": [
{
"name": "BCA",
"account_number": 1487332
},
{
"name": "BRI",
"account_number": 5452730
}
]
}
},
{
"_index": "customers",
"_id": "4LJGYowBtsOVXdO2r8HA",
"_score": 2.5432863,
"_source": {
"username": "username476",
"first_name": "Dinny",
"last_name": "McMillan",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1998-03-25",
"address": {
"street": "56 Harper Court",
"city": "Pensacola",
"province": "Florida",
"country": "United States",
"zip_code": "32595"
},
"hobbies": [
"Basket",
"Reading"
],
"banks": [
{
"name": "BRI",
"account_number": 9892239
},
{
"name": "BCA",
"account_number": 8727644
}
]
}
},
{
"_index": "customers",
"_id": "urJGYowBtsOVXdO2sMIZ",
"_score": 2.5432863,
"_source": {
"username": "username585",
"first_name": "Emogene",
"last_name": "Hupka",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1985-12-03",
"address": {
"street": "92 Bay Lane",
"city": "Springfield",
"province": "Massachusetts",
"country": "United States",
"zip_code": "01129"
},
"hobbies": [
"Basket",
"Reading"
],
"banks": [
{
"name": "BRI",
"account_number": 8741448
},
{
"name": "BCA",
"account_number": 8658356
}
]
}
},
{
"_index": "customers",
"_id": "hrJGYowBtsOVXdO2sMMa",
"_score": 2.5432863,
"_source": {
"username": "username687",
"first_name": "Victor",
"last_name": "Jinkin",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1982-01-05",
"address": {
"street": "8 Trailsway Center",
"city": "Brooklyn",
"province": "New York",
"country": "United States",
"zip_code": "11205"
},
"hobbies": [
"Basket",
"Gaming"
],
"banks": [
{
"name": "BRI",
"account_number": 3352823
},
{
"name": "BCA",
"account_number": 4278436
}
]
}
},
{
"_index": "customers",
"_id": "9rJGYowBtsOVXdO2sMMa",
"_score": 2.5432863,
"_source": {
"username": "username743",
"first_name": "Maurita",
"last_name": "Seelbach",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1986-01-24",
"address": {
"street": "80357 Clove Court",
"city": "Houston",
"province": "Texas",
"country": "United States",
"zip_code": "77065"
},
"hobbies": [
"Soccer",
"Gaming"
],
"banks": [
{
"name": "BCA",
"account_number": 3299728
},
{
"name": "BRI",
"account_number": 8314066
}
]
}
},
{
"_index": "customers",
"_id": "KLJGYowBtsOVXdO2sMSe",
"_score": 2.5432863,
"_source": {
"username": "username768",
"first_name": "Valeria",
"last_name": "Valder",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1984-10-06",
"address": {
"street": "9219 North Circle",
"city": "Whittier",
"province": "California",
"country": "United States",
"zip_code": "90610"
},
"hobbies": [
"Soccer",
"Reading"
],
"banks": [
{
"name": "BRI",
"account_number": 4368607
},
{
"name": "BCA",
"account_number": 6427897
}
]
}
},
{
"_index": "customers",
"_id": "tLJGYowBtsOVXdO2sMSg",
"_score": 2.5432863,
"_source": {
"username": "username838",
"first_name": "Dorey",
"last_name": "Betun",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1971-01-05",
"address": {
"street": "7696 Sutherland Hill",
"city": "Newark",
"province": "New Jersey",
"country": "United States",
"zip_code": "07104"
},
"hobbies": [
"Reading",
"Soccer"
],
"banks": [
{
"name": "BCA",
"account_number": 6836664
},
{
"name": "BRI",
"account_number": 8031975
}
]
}
}
]
}
}Minimum Should Match
- What happens when we use a
Boolean Queryand only useshould, withoutmustandfilter? By default, at least one condition withinshouldhas to match with the document. - What happens when we add
mustorfilterto ourBoolean Query? The minimumshouldmatch value becomes 0, which meansshouldis not mandatory. - What if we want to change the minimum
shouldmatch value? We have to increase the value for the minimumshouldmatch.
Boolean Query - Must dan Should
# POST http://localhost:9200/customers/_search
response = es.search(index='customers', body={
'query': {
'bool': {
'must': {
'term': {
'hobbies': 'gaming'
}
},
'should': [
{
'term': {
'banks.name': 'bca'
}
},
{
'term': {
'banks.name': 'bri'
}
}
]
}
}
})
print(json.dumps(response.body, indent=4)){
"took": 3,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 321,
"relation": "eq"
},
"max_score": 3.6790433,
"hits": [
{
"_index": "customers",
"_id": "cLJGYowBtsOVXdO2r74I",
"_score": 3.6790433,
"_source": {
"username": "username36",
"first_name": "Catlee",
"last_name": "Ivy",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1994-08-24",
"address": {
"street": "64347 7th Parkway",
"city": "New Orleans",
"province": "Louisiana",
"country": "United States",
"zip_code": "70149"
},
"hobbies": [
"Gaming",
"Soccer"
],
"banks": [
{
"name": "BCA",
"account_number": 5844642
},
{
"name": "BRI",
"account_number": 8309997
}
]
}
},
{
"_index": "customers",
"_id": "hrJGYowBtsOVXdO2sMMa",
"_score": 3.6790433,
"_source": {
"username": "username687",
"first_name": "Victor",
"last_name": "Jinkin",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1982-01-05",
"address": {
"street": "8 Trailsway Center",
"city": "Brooklyn",
"province": "New York",
"country": "United States",
"zip_code": "11205"
},
"hobbies": [
"Basket",
"Gaming"
],
"banks": [
{
"name": "BRI",
"account_number": 3352823
},
{
"name": "BCA",
"account_number": 4278436
}
]
}
},
{
"_index": "customers",
"_id": "9rJGYowBtsOVXdO2sMMa",
"_score": 3.6790433,
"_source": {
"username": "username743",
"first_name": "Maurita",
"last_name": "Seelbach",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1986-01-24",
"address": {
"street": "80357 Clove Court",
"city": "Houston",
"province": "Texas",
"country": "United States",
"zip_code": "77065"
},
"hobbies": [
"Soccer",
"Gaming"
],
"banks": [
{
"name": "BCA",
"account_number": 3299728
},
{
"name": "BRI",
"account_number": 8314066
}
]
}
},
{
"_index": "customers",
"_id": "PLJGYowBtsOVXdO2r8HA",
"_score": 3.6396484,
"_source": {
"username": "username394",
"first_name": "Linoel",
"last_name": "Kopec",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1991-03-28",
"address": {
"street": "3 Kipling Junction",
"city": "Gainesville",
"province": "Florida",
"country": "United States",
"zip_code": "32627"
},
"hobbies": [
"Traveling",
"Gaming"
],
"banks": [
{
"name": "BCA",
"account_number": 3872364
},
{
"name": "BRI",
"account_number": 5641432
},
{
"name": "BCA",
"account_number": 2723248
}
]
}
},
{
"_index": "customers",
"_id": "6rJGYowBtsOVXdO2r8HA",
"_score": 3.6396484,
"_source": {
"username": "username481",
"first_name": "Juanita",
"last_name": "Lansdale",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1995-10-26",
"address": {
"street": "4 Sunfield Lane",
"city": "Naples",
"province": "Florida",
"country": "United States",
"zip_code": "34114"
},
"hobbies": [
"Traveling",
"Gaming"
],
"banks": [
{
"name": "BCA",
"account_number": 7122066
},
{
"name": "BCA",
"account_number": 4992498
},
{
"name": "BRI",
"account_number": 8892123
}
]
}
},
{
"_index": "customers",
"_id": "ZLJGYowBtsOVXdO2r8HA",
"_score": 3.334869,
"_source": {
"username": "username414",
"first_name": "Hunter",
"last_name": "Qualtro",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1976-08-06",
"address": {
"street": "57 Cordelia Avenue",
"city": "Wilmington",
"province": "North Carolina",
"country": "United States",
"zip_code": "28410"
},
"hobbies": [
"Soccer",
"Gaming"
],
"banks": [
{
"name": "BCA Digital",
"account_number": 9464981
},
{
"name": "BRI",
"account_number": 2065565
},
{
"name": "BCA",
"account_number": 4272004
}
]
}
},
{
"_index": "customers",
"_id": "bLJGYowBtsOVXdO2r74I",
"_score": 3.268033,
"_source": {
"username": "username34",
"first_name": "Salomo",
"last_name": "Stubbington",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1974-08-09",
"address": {
"street": "9297 Dorton Plaza",
"city": "El Paso",
"province": "Texas",
"country": "United States",
"zip_code": "79940"
},
"hobbies": [
"Coding",
"Gaming"
],
"banks": [
{
"name": "BRI",
"account_number": 6392777
},
{
"name": "BCA Digital",
"account_number": 8157580
}
]
}
},
{
"_index": "customers",
"_id": "XLJGYowBtsOVXdO2r78K",
"_score": 3.268033,
"_source": {
"username": "username154",
"first_name": "Lillian",
"last_name": "Loukes",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1983-12-24",
"address": {
"street": "056 Nancy Drive",
"city": "Miami",
"province": "Florida",
"country": "United States",
"zip_code": "33129"
},
"hobbies": [
"Coding",
"Gaming"
],
"banks": [
{
"name": "BRI",
"account_number": 2485419
},
{
"name": "BSI",
"account_number": 7284668
},
{
"name": "BCA",
"account_number": 9927406
}
]
}
},
{
"_index": "customers",
"_id": "grJGYowBtsOVXdO2r78K",
"_score": 3.268033,
"_source": {
"username": "username173",
"first_name": "Blakeley",
"last_name": "Adhams",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1990-07-05",
"address": {
"street": "98 Springs Way",
"city": "Baton Rouge",
"province": "Louisiana",
"country": "United States",
"zip_code": "70826"
},
"hobbies": [
"Gaming",
"Traveling"
],
"banks": [
{
"name": "BRI",
"account_number": 4077771
},
{
"name": "Mandiri",
"account_number": 7632202
},
{
"name": "BCA",
"account_number": 9874926
}
]
}
},
{
"_index": "customers",
"_id": "5rJGYowBtsOVXdO2r8C_",
"_score": 3.268033,
"_source": {
"username": "username351",
"first_name": "Merissa",
"last_name": "Pettegre",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1998-01-31",
"address": {
"street": "5 Dovetail Alley",
"city": "Allentown",
"province": "Pennsylvania",
"country": "United States",
"zip_code": "18105"
},
"hobbies": [
"Soccer",
"Gaming"
],
"banks": [
{
"name": "BRI",
"account_number": 8494794
},
{
"name": "BCA Digital",
"account_number": 6848718
}
]
}
}
]
}
}Boolean Query - Minimum Should Match
# POST http://localhost:9200/customers/_search
response = es.search(index='customers', body={
'query': {
'bool': {
'must': {
'term': {
'hobbies': 'gaming'
}
},
'should': [
{
'term': {
'banks.name': 'bca'
}
},
{
'term': {
'banks.name': 'bri'
}
}
],
'minimum_should_match': 1
}
}
})
print(json.dumps(response.body, indent=4)){
"took": 3,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 159,
"relation": "eq"
},
"max_score": 3.6790433,
"hits": [
{
"_index": "customers",
"_id": "cLJGYowBtsOVXdO2r74I",
"_score": 3.6790433,
"_source": {
"username": "username36",
"first_name": "Catlee",
"last_name": "Ivy",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1994-08-24",
"address": {
"street": "64347 7th Parkway",
"city": "New Orleans",
"province": "Louisiana",
"country": "United States",
"zip_code": "70149"
},
"hobbies": [
"Gaming",
"Soccer"
],
"banks": [
{
"name": "BCA",
"account_number": 5844642
},
{
"name": "BRI",
"account_number": 8309997
}
]
}
},
{
"_index": "customers",
"_id": "hrJGYowBtsOVXdO2sMMa",
"_score": 3.6790433,
"_source": {
"username": "username687",
"first_name": "Victor",
"last_name": "Jinkin",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1982-01-05",
"address": {
"street": "8 Trailsway Center",
"city": "Brooklyn",
"province": "New York",
"country": "United States",
"zip_code": "11205"
},
"hobbies": [
"Basket",
"Gaming"
],
"banks": [
{
"name": "BRI",
"account_number": 3352823
},
{
"name": "BCA",
"account_number": 4278436
}
]
}
},
{
"_index": "customers",
"_id": "9rJGYowBtsOVXdO2sMMa",
"_score": 3.6790433,
"_source": {
"username": "username743",
"first_name": "Maurita",
"last_name": "Seelbach",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1986-01-24",
"address": {
"street": "80357 Clove Court",
"city": "Houston",
"province": "Texas",
"country": "United States",
"zip_code": "77065"
},
"hobbies": [
"Soccer",
"Gaming"
],
"banks": [
{
"name": "BCA",
"account_number": 3299728
},
{
"name": "BRI",
"account_number": 8314066
}
]
}
},
{
"_index": "customers",
"_id": "PLJGYowBtsOVXdO2r8HA",
"_score": 3.6396484,
"_source": {
"username": "username394",
"first_name": "Linoel",
"last_name": "Kopec",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1991-03-28",
"address": {
"street": "3 Kipling Junction",
"city": "Gainesville",
"province": "Florida",
"country": "United States",
"zip_code": "32627"
},
"hobbies": [
"Traveling",
"Gaming"
],
"banks": [
{
"name": "BCA",
"account_number": 3872364
},
{
"name": "BRI",
"account_number": 5641432
},
{
"name": "BCA",
"account_number": 2723248
}
]
}
},
{
"_index": "customers",
"_id": "6rJGYowBtsOVXdO2r8HA",
"_score": 3.6396484,
"_source": {
"username": "username481",
"first_name": "Juanita",
"last_name": "Lansdale",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1995-10-26",
"address": {
"street": "4 Sunfield Lane",
"city": "Naples",
"province": "Florida",
"country": "United States",
"zip_code": "34114"
},
"hobbies": [
"Traveling",
"Gaming"
],
"banks": [
{
"name": "BCA",
"account_number": 7122066
},
{
"name": "BCA",
"account_number": 4992498
},
{
"name": "BRI",
"account_number": 8892123
}
]
}
},
{
"_index": "customers",
"_id": "ZLJGYowBtsOVXdO2r8HA",
"_score": 3.334869,
"_source": {
"username": "username414",
"first_name": "Hunter",
"last_name": "Qualtro",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1976-08-06",
"address": {
"street": "57 Cordelia Avenue",
"city": "Wilmington",
"province": "North Carolina",
"country": "United States",
"zip_code": "28410"
},
"hobbies": [
"Soccer",
"Gaming"
],
"banks": [
{
"name": "BCA Digital",
"account_number": 9464981
},
{
"name": "BRI",
"account_number": 2065565
},
{
"name": "BCA",
"account_number": 4272004
}
]
}
},
{
"_index": "customers",
"_id": "bLJGYowBtsOVXdO2r74I",
"_score": 3.268033,
"_source": {
"username": "username34",
"first_name": "Salomo",
"last_name": "Stubbington",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1974-08-09",
"address": {
"street": "9297 Dorton Plaza",
"city": "El Paso",
"province": "Texas",
"country": "United States",
"zip_code": "79940"
},
"hobbies": [
"Coding",
"Gaming"
],
"banks": [
{
"name": "BRI",
"account_number": 6392777
},
{
"name": "BCA Digital",
"account_number": 8157580
}
]
}
},
{
"_index": "customers",
"_id": "XLJGYowBtsOVXdO2r78K",
"_score": 3.268033,
"_source": {
"username": "username154",
"first_name": "Lillian",
"last_name": "Loukes",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1983-12-24",
"address": {
"street": "056 Nancy Drive",
"city": "Miami",
"province": "Florida",
"country": "United States",
"zip_code": "33129"
},
"hobbies": [
"Coding",
"Gaming"
],
"banks": [
{
"name": "BRI",
"account_number": 2485419
},
{
"name": "BSI",
"account_number": 7284668
},
{
"name": "BCA",
"account_number": 9927406
}
]
}
},
{
"_index": "customers",
"_id": "grJGYowBtsOVXdO2r78K",
"_score": 3.268033,
"_source": {
"username": "username173",
"first_name": "Blakeley",
"last_name": "Adhams",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1990-07-05",
"address": {
"street": "98 Springs Way",
"city": "Baton Rouge",
"province": "Louisiana",
"country": "United States",
"zip_code": "70826"
},
"hobbies": [
"Gaming",
"Traveling"
],
"banks": [
{
"name": "BRI",
"account_number": 4077771
},
{
"name": "Mandiri",
"account_number": 7632202
},
{
"name": "BCA",
"account_number": 9874926
}
]
}
},
{
"_index": "customers",
"_id": "5rJGYowBtsOVXdO2r8C_",
"_score": 3.268033,
"_source": {
"username": "username351",
"first_name": "Merissa",
"last_name": "Pettegre",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1998-01-31",
"address": {
"street": "5 Dovetail Alley",
"city": "Allentown",
"province": "Pennsylvania",
"country": "United States",
"zip_code": "18105"
},
"hobbies": [
"Soccer",
"Gaming"
],
"banks": [
{
"name": "BRI",
"account_number": 8494794
},
{
"name": "BCA Digital",
"account_number": 6848718
}
]
}
}
]
}
}Boolean Query - Minimum Should Match 2
# POST http://localhost:9200/customers/_search
response = es.search(index='customers', body={
'query': {
'bool': {
'must': {
'term': {
'hobbies': 'gaming'
}
},
'should': [
{
'term': {
'banks.name': 'bca'
}
},
{
'term': {
'banks.name': 'bri'
}
}
],
'minimum_should_match': 2
}
}
})
print(json.dumps(response.body, indent=4)){
"took": 3,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 19,
"relation": "eq"
},
"max_score": 3.6790433,
"hits": [
{
"_index": "customers",
"_id": "cLJGYowBtsOVXdO2r74I",
"_score": 3.6790433,
"_source": {
"username": "username36",
"first_name": "Catlee",
"last_name": "Ivy",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1994-08-24",
"address": {
"street": "64347 7th Parkway",
"city": "New Orleans",
"province": "Louisiana",
"country": "United States",
"zip_code": "70149"
},
"hobbies": [
"Gaming",
"Soccer"
],
"banks": [
{
"name": "BCA",
"account_number": 5844642
},
{
"name": "BRI",
"account_number": 8309997
}
]
}
},
{
"_index": "customers",
"_id": "hrJGYowBtsOVXdO2sMMa",
"_score": 3.6790433,
"_source": {
"username": "username687",
"first_name": "Victor",
"last_name": "Jinkin",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1982-01-05",
"address": {
"street": "8 Trailsway Center",
"city": "Brooklyn",
"province": "New York",
"country": "United States",
"zip_code": "11205"
},
"hobbies": [
"Basket",
"Gaming"
],
"banks": [
{
"name": "BRI",
"account_number": 3352823
},
{
"name": "BCA",
"account_number": 4278436
}
]
}
},
{
"_index": "customers",
"_id": "9rJGYowBtsOVXdO2sMMa",
"_score": 3.6790433,
"_source": {
"username": "username743",
"first_name": "Maurita",
"last_name": "Seelbach",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1986-01-24",
"address": {
"street": "80357 Clove Court",
"city": "Houston",
"province": "Texas",
"country": "United States",
"zip_code": "77065"
},
"hobbies": [
"Soccer",
"Gaming"
],
"banks": [
{
"name": "BCA",
"account_number": 3299728
},
{
"name": "BRI",
"account_number": 8314066
}
]
}
},
{
"_index": "customers",
"_id": "PLJGYowBtsOVXdO2r8HA",
"_score": 3.6396487,
"_source": {
"username": "username394",
"first_name": "Linoel",
"last_name": "Kopec",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1991-03-28",
"address": {
"street": "3 Kipling Junction",
"city": "Gainesville",
"province": "Florida",
"country": "United States",
"zip_code": "32627"
},
"hobbies": [
"Traveling",
"Gaming"
],
"banks": [
{
"name": "BCA",
"account_number": 3872364
},
{
"name": "BRI",
"account_number": 5641432
},
{
"name": "BCA",
"account_number": 2723248
}
]
}
},
{
"_index": "customers",
"_id": "6rJGYowBtsOVXdO2r8HA",
"_score": 3.6396487,
"_source": {
"username": "username481",
"first_name": "Juanita",
"last_name": "Lansdale",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1995-10-26",
"address": {
"street": "4 Sunfield Lane",
"city": "Naples",
"province": "Florida",
"country": "United States",
"zip_code": "34114"
},
"hobbies": [
"Traveling",
"Gaming"
],
"banks": [
{
"name": "BCA",
"account_number": 7122066
},
{
"name": "BCA",
"account_number": 4992498
},
{
"name": "BRI",
"account_number": 8892123
}
]
}
},
{
"_index": "customers",
"_id": "ZLJGYowBtsOVXdO2r8HA",
"_score": 3.334869,
"_source": {
"username": "username414",
"first_name": "Hunter",
"last_name": "Qualtro",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1976-08-06",
"address": {
"street": "57 Cordelia Avenue",
"city": "Wilmington",
"province": "North Carolina",
"country": "United States",
"zip_code": "28410"
},
"hobbies": [
"Soccer",
"Gaming"
],
"banks": [
{
"name": "BCA Digital",
"account_number": 9464981
},
{
"name": "BRI",
"account_number": 2065565
},
{
"name": "BCA",
"account_number": 4272004
}
]
}
},
{
"_index": "customers",
"_id": "bLJGYowBtsOVXdO2r74I",
"_score": 3.268033,
"_source": {
"username": "username34",
"first_name": "Salomo",
"last_name": "Stubbington",
"email": "[email protected]",
"gender": "Male",
"birth_date": "1974-08-09",
"address": {
"street": "9297 Dorton Plaza",
"city": "El Paso",
"province": "Texas",
"country": "United States",
"zip_code": "79940"
},
"hobbies": [
"Coding",
"Gaming"
],
"banks": [
{
"name": "BRI",
"account_number": 6392777
},
{
"name": "BCA Digital",
"account_number": 8157580
}
]
}
},
{
"_index": "customers",
"_id": "XLJGYowBtsOVXdO2r78K",
"_score": 3.268033,
"_source": {
"username": "username154",
"first_name": "Lillian",
"last_name": "Loukes",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1983-12-24",
"address": {
"street": "056 Nancy Drive",
"city": "Miami",
"province": "Florida",
"country": "United States",
"zip_code": "33129"
},
"hobbies": [
"Coding",
"Gaming"
],
"banks": [
{
"name": "BRI",
"account_number": 2485419
},
{
"name": "BSI",
"account_number": 7284668
},
{
"name": "BCA",
"account_number": 9927406
}
]
}
},
{
"_index": "customers",
"_id": "grJGYowBtsOVXdO2r78K",
"_score": 3.268033,
"_source": {
"username": "username173",
"first_name": "Blakeley",
"last_name": "Adhams",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1990-07-05",
"address": {
"street": "98 Springs Way",
"city": "Baton Rouge",
"province": "Louisiana",
"country": "United States",
"zip_code": "70826"
},
"hobbies": [
"Gaming",
"Traveling"
],
"banks": [
{
"name": "BRI",
"account_number": 4077771
},
{
"name": "Mandiri",
"account_number": 7632202
},
{
"name": "BCA",
"account_number": 9874926
}
]
}
},
{
"_index": "customers",
"_id": "5rJGYowBtsOVXdO2r8C_",
"_score": 3.268033,
"_source": {
"username": "username351",
"first_name": "Merissa",
"last_name": "Pettegre",
"email": "[email protected]",
"gender": "Female",
"birth_date": "1998-01-31",
"address": {
"street": "5 Dovetail Alley",
"city": "Allentown",
"province": "Pennsylvania",
"country": "United States",
"zip_code": "18105"
},
"hobbies": [
"Soccer",
"Gaming"
],
"banks": [
{
"name": "BRI",
"account_number": 8494794
},
{
"name": "BCA Digital",
"account_number": 6848718
}
]
}
}
]
}
}Cat API
Imagine Elasticsearch as a bustling city filled with information. The Compact and Aligned Text (CAT) APIs are like the city’s information kiosks, where you can get a snapshot of the city’s health, districts (indices), buildings (shards), citizens (nodes), and more.
These CAT APIs are not typically used for day-to-day city life (the applications we build), but rather for city planners and maintenance crews (users maintaining Elasticsearch).
There is a vast network of these information kiosks (APIs) throughout the city. You can see them all by visiting the city’s main information center (GET /_cat). If you’d like to learn more about these kiosks, you can visit the city’s online guidebook at Elasticsearch’s CAT APIs documentation.
import requests
response = requests.get('http://localhost:9200/_cat')
print(response.text)=^.^=
/_cat/allocation
/_cat/shards
/_cat/shards/{index}
/_cat/master
/_cat/nodes
/_cat/tasks
/_cat/indices
/_cat/indices/{index}
/_cat/segments
/_cat/segments/{index}
/_cat/count
/_cat/count/{index}
/_cat/recovery
/_cat/recovery/{index}
/_cat/health
/_cat/pending_tasks
/_cat/aliases
/_cat/aliases/{alias}
/_cat/thread_pool
/_cat/thread_pool/{thread_pools}
/_cat/plugins
/_cat/fielddata
/_cat/fielddata/{fields}
/_cat/nodeattrs
/_cat/repositories
/_cat/snapshots/{repository}
/_cat/templates
/_cat/component_templates/_cat/ml/anomaly_detectors
/_cat/ml/anomaly_detectors/{job_id}
/_cat/ml/datafeeds
/_cat/ml/datafeeds/{datafeed_id}
/_cat/ml/trained_models
/_cat/ml/trained_models/{model_id}
/_cat/ml/data_frame/analytics
/_cat/ml/data_frame/analytics/{id}
/_cat/transforms
/_cat/transforms/{transform_id}
response = requests.get('http://localhost:9200/_cat/health')
print(response.text)1702458158 09:02:38 ai-bootcamp yellow 1 1 3 3 0 0 3 0 - 50.0%
response = requests.get('http://localhost:9200/_cat/indices')
print(response.text)yellow open orders iZ5yo0jhQzen6WpHWu02pw 1 1 1 0 6.6kb 6.6kb
yellow open customers DvA3r0qSTxSDO5lJaQNquQ 1 1 2000 0 555.7kb 555.7kb
yellow open products 1cdU4XybTUad8p0TyqKtlQ 1 1 5 0 5.5kb 5.5kb