Filtering/Query language in Pyxis#

Endpoints that return a list of objects support filtering using a filtering language. The filter query has to be passed in the filter query parameter. Filtering language is based on RSQL with few added functions. Compressed fields cannot be used in RSQL expressions.

Grammar and semantic#

RSQL expression is composed of one or more comparisons related to each other with logical operators:

  • Logical AND: ; or and

  • Logical OR: , or or

  • Logical NOR: nor

By default, the AND operator takes precedence (i.e., it’s evaluated before any OR or NOR operators are). However, a parenthesized expression can be used to change the precedence, yielding whatever the contained expression yields.

comp_1 or comp_2 and ( comp_3 or comp_4 )
comp_1 or comp_2 and comp_3_4
comp_1 or comp_2_3_4
comp_1_2_3_4

A comparison is composed of a field name, a comparison operator, and an argument/value.

comparison = <field name> <comparison op> <value>

We recommend URL-encoding your filter queries as special characters can cause confusing and hard-to-debug issues.

NOT#

The RSQL implementation in the Pyxis REST also supports NOT operation as a unary operation.

not( field == value )
not( comp_1 or comp_2 )
not( comp_1 or not( comp_2 ))

Field names#

The selector identifies a field of the object representation to filter by. It can be any non-empty Unicode string that contains only letters, dots, numbers, and underscores. Pyxis API will return HTTP 400 if either the field does not exist or the field cannot be used for filtering.

Nested fields and fields of objects in a list are available through the use of a dot, i.e., nested_object.name and list.name. That is similar to how these fields can be referenced in MongoDB.

Comparison operators#

Comparison operators are in FIQL notation, and some of them have an alternative syntax as well:

  • Equal to: ==

  • Not equal to: !=

  • Less than : =lt= or <

  • Less than or equal to : =le= or <=

  • Greater than operator : =gt= or >

  • Greater than or equal to : =ge= or >=

  • In: =in=

  • Not in: =out=

  • All: =all=

  • Regex: ~= or =regex=

  • Size: =size=

  • Case insensitive regex: =iregex=

  • ElemMatch: =em=

The comparison operators work the same way as their MongoDB counterparts, as we transform the user filter into an equivalent MongoDB filter.

Comparison operators work in unintuitive ways when used on lists. Please look at the Lists section before filtering on lists.

Arguments#

An argument can be a single value or multiple values in parenthesis separated by a comma. A value that doesn’t contain any reserved character ((, )) or white space can be unquoted, and other values must be enclosed in single or double quotes.

We strongly recommend using quoted strings to pass parameters, especially regexes, as the unquoted strings only allow selected characters. By quoting a string parameter, you can also protect it from being retyped, as the parameter that only contains integers can be retyped to integer.

You can see the allowed character by checking the selection regex [a-zA-Z0-9_\-*\[\]^$:\|+.\/}{?@&<>%®~#!^]+. On the other hand, the quoted strings match all the characters between two unescaped quotes. The regexes we used to match quoted strings are \".*?(?<!\\\\)\" and \'.*?(?<!\\\\)\'.

If you need to use both single and double quotes inside a quoted argument, then you must escape one of them using \ (backslash). If you want to use \ literally, then double it as \\.

Dates and Datetimes#

Pyxis API uses ISO 8601 format for dates and datetimes. Here are a few examples of dates Pyxis API will accept:

  • 2022-01

  • 2022-01-01

  • 2022-W02

  • 2022-W02-5

  • 2022-W02-5T10:55

  • 2022-W02-5T10:55:17

Lists#

Lists have several special properties that are further explained in this section. Especially lists of objects can work unintuitively, so you should read this section if you intend to filter on lists of objects.

Element match vs comparison operators#

When filtering on a list of objects, you have two options, element match or chain of comparison operators chained by logical operators.

The element match operator will check if at least one object inside a list satisfies all the requirements within the element match definition.

On the contrary, the chain of comparison operators checks if at least one object within the list satisfies that particular comparison, but there does not have to be an object that meets all of them at once.

That can cause a lot of confusion and will be explained in the example. Imagine that we have this list called LIST:

  • {A: 1, B: 1}

  • {A: 2, B: 2}

  • {A: 3, B: 3}

The filter query LIST.A==1 AND LIST.B==2 will match this list, as there is one object where A has a value of 1 and an object where B has a value of 2.

The filter query LIST=em=(A==1 and B==2) will not match this list as there is no object where A equals one at the same time as B equals 2.

The filter query LIST=em=(A==1 and B==1) will match this list, as there is an object where A equals one at the same time as B equals 1.

Filtering by index#

You can create a query to check if an object on a specific position within a list satisfies your requirements. It is done in the same way as in MongoDB queries, i.e., by putting position into the field name, for example, array.1 (list of scalars) or array.1.key (array of objects).

For example, array.1==2 checks if a second member in the list, MongoDB lists start from 0, equals two. Query array.1.key==2 checks if the second object in the list contains a key called key with value two.

List size#

You can also use array size in your queries. Simply add # as a suffix to the array field name. For example, if you need a list of product listings that contain more than one contact, you would create a query contacts#>=1. Please note that the # in the query has to be URL encoded, so the filter will look like contacts%23>=1 in the URL.

Examples#

Examples of RSQL expressions in both FIQL-like and alternative notation:

- name=="Kill Bill";year=gt=2003
- name=="Kill Bill" and year>2003
- director.name=='Christopher Nolan' and director.gender==male
- genres=in=(sci-fi,action);(director=='Christopher Nolan',actor=regex=*Bale);year=ge=2000
- genres=in=(sci-fi,action) and (director=='Christopher Nolan' or actor~=*Bale) and year>=2000
- director.lastName==Nolan;year=ge=2000;year=lt=2010
- director.lastName==Nolan and year>=2000 and year<2010
- genres=in=(sci-fi,action);genres=out=(romance,animated,horror),director==Que*Tarantino
- genres=in=(sci-fi,action) and genres=out=(romance,animated,horror) or director==Que*Tarantino
- items=em=(name=='John' and age==20)