Query Search
Query searches help you search the contents of forms attached to your topics, as well as the values of other meta-data attached to the topic. Using query searches you can search:
- The fields of forms
- Parent relationships
- File attachment information (but not the attached files themselves)
Query searches are defined using a simple query language. The language consists of
field specifiers and
constants joined with
operators.
Field specifiers
You use field specifiers to say what value from the topic you are interested
in.
All meta-data in a topic is referenced according to a simple plan.
-
name
- name of the topic
-
web
- name of the web the topic is within
-
text
- the body text of the topic (without embedded meta-data)
-
META:FILEATTACHMENT
- for each attachment
-
name
-
attr
-
path
-
size
-
user
-
rev
-
date
-
comment
-
META:TOPICPARENT
-
META:TOPICINFO
-
author
-
date
-
format
-
version
- topic version (integer)
-
META:TOPICMOVED
-
META:FORM
- the main form of the topic
-
META:FIELD
- the fields in the form.
- for each field in the form
-
name
- name of the field
-
title
- title of the field
-
value
- what is stored in the field
-
form
- name of the form the field is in (currently always equal to META:FORM.name)
-
attributes
- string stored in the attributes, like H
for hidden
-
META:PREFERENCE
- for each preference in the topic
See
MetaData for details of what all these entries mean. Note that the set
of meta-data types (and the aliases used to refer to them) may be extended
by Foswiki extensions.
Most things at the top level of the plan -
META:TOPICPARENT
,
META:TOPICINFO
etc - are
structures which are indexed by
keys. For example,
META:TOPICINFO
has 4 entries, which are indexed by the keys
author
,
date
,
format
and
version
.
META:FILEATTACHMENT
,
META:FIELD
and
META:PREFERENCE
are all
arrays, which means they can have any number of records under them. Arrays are indexed by
numbers - for example, the first entry in the
META:FIELD
array is entry 0.
It's a bit clumsy having to type
META:FILEATTACHMENT
every time you want to refer to the array of attachments in a topic, so there are some predefined aliases that make it a bit less typing:
-
attachments
means the same as META:FILEATTACHMENT
-
info
means the same as META:TOPICINFO
-
parent
means the same as META:TOPICPARENT
. Note: parent
is itself a map; use parent.name
to access the name of the parent topic
-
moved
means the same as META:TOPICMOVED
-
form
means the same as META:FORM
, so to test if a topic has a form named 'UserForm' you test for "form.name ~ '*.UserForm'"
-
fields
means the same as META:FIELD
, You can also use the name of the form (the value of form.name
e.g. PersonForm
)
-
preferences
means the same as META:PREFERENCE
- extensions may add additional aliases when they register new meta-data types
Fields in this plan are referenced using a simple
field specifier syntax:
Syntax |
Means |
Examples |
X |
refers to the field named X . |
info , META:TOPICMOVED , attachments , name . |
X.Y |
refers to the entry with the key Y in the structure named X . If X is an array of structure, then returns an array made up from the Y entry of each member of the array. |
info.date , moved.by , META:TOPICPARENT.name , attachments.name |
X[query] |
refers to all the elements of the array X that match query. If query is of the form name='Y' then you can use the same X.Y syntax as is used for accessing structures. |
attachments[size>1024] , DocumentContainer[name!='Summary' AND value~'top secret'].value |
X[N] |
where X is an array and N is an integer number >= 0, gets the Nth element of the array X . If N is a floating point number, the integer part will be used as the index. Negative indices can be used to index the array from the end e.g. attachments[-1] to get the last attachment. |
attachments[3] |
X/Y |
accesses Y from the topic specified by the value of X . X must evaluate to a topic name |
parent.name/(form.name='ExampleForm') will evaluate to true if (1) the topic has a parent, (2) the parent topic has the main form type ExampleForm . |
{X} |
expands to the value of the configure setting {X}, if it is accessible, or '' otherwise |
only some configuration settings are available: {ScriptSuffix} , {LoginManager} , {AuthScripts} , {LoginNameFilterIn} , {AdminUserLogin} , {AdminUserWikiName} , {SuperAdminGroup} , {UsersTopicName} , {AuthRealm} , {MinPasswordLength} , {Register}{AllowLoginName} , {Register}{EnableNewUserRegistration} , {Register}{NeedVerification} , {Register}{RegistrationAgentWikiName} , {AllowInlineScript} , {DenyDotDotInclude} , {UploadFilter} , {NameFilter} , {AccessibleCFG} , {AntiSpam}{EmailPadding} , {AntiSpam}{EntityEncode} , {AntiSpam}{HideUserDetails} , {AntiSpam}{RobotsAreWelcome} , {Stats}{TopViews} , {Stats}{TopContrib} , {Stats}{TopicName} , {UserInterfaceInternationalisation} , {UseLocale} , {Site}{Locale} , {Site}{CharSet} , {DisplayTimeValues} , {DefaultDateFormat} , {Site}{LocaleRegexes} , {UpperNational} , {LowerNational} , {PluralToSingular} , {EnableHierarchicalWebs} , {WebMasterEmail} , {WebMasterName} , {NotifyTopicName} , {SystemWebName} , {TrashWebName} , {SitePrefsTopicName} , {LocalSitePreferences} , {HomeTopicName} , {WebPrefsTopicName} , {UsersWebName} , {TemplatePath} , {LinkProtocolPattern} , {NumberOfRevisions} , {MaxRevisionsInADiff} , {ReplaceIfEditedAgainWithin} , {LeaseLength} , {LeaseLengthLessForceful} , {Plugins}{WebSearchPath} , {PluginsOrder} , {Cache}{Enabled} , {Validation}{Method} , {Register}{DisablePasswordConfirmation} |
Note: at some point Foswiki may support multiple forms in the same topic. For this reason you are recommended
not to use the
fields
shortcut when accessing form fields, but always use the name of the form instead.
If you use the name of a field (for example,
LastName
) in the query without a . before it, that is taken to mean "the value of the field named this". This works if and only if the field name isn't the same as of the top level entry names or their aliases described above. For example, the following expressions will all evaluate to the same thing:
-
PersonForm[name='Lastname'].value
-
Lastname
-
PersonForm.Lastname
If
X
would conflict with the name of an entry or alias (e.g. it's
moved
or maybe
parent
), you can prepend the name of the form followed by a dot, as shown in the last example.
Constants
You use constants for the values that you compare with fields. Constants are either strings, or numbers.
String Constants
String constants are always delimited by single-quotes. You can use backslash
\
to include the following special characters:
Code |
Meaning |
\n |
newline |
\t |
tab |
\033 |
octal character code |
=\x7f |
hexadecimal character code |
=\x{1234} |
heaxadecimal wide character code |
\\ |
a single \ |
All other occurrences of backslashes are carried through into the string, so
\d
means
\d
(unless the string is used as a regular expression, in which case it means any digit).
Numerical constants
Numbers can be any signed or unsigned integer or floating point number using standard scientific notation e.g. -1.2e-3 represents -0.0012
Operators
Field specifiers and constants are combined using
operators to create queries.
Operator |
Meaning |
= |
Left-hand side (LHS) exactly matches the value on the Right-hand side (RHS). Numbers and strings can be compared. |
!= |
Inverse of = . |
~ |
wildcard match ('*' will match any number of characters, '?' will match any single character e.g. "PersonForm.Surname ~ '*Smit?'") Note: Surname ~ 'Smith' is the same as Surname = 'Smith' |
=~ |
regular expression match, see RegularExpressions for details. |
< |
LHS is less than RHS. If both sides are numbers, the order is numeric. Otherwise it is lexical (applies to all comparison operators) |
> |
greater than |
<= |
less than or equal to |
>= |
greater than or equal to |
lc(x) |
Converts x to lower case, Use for caseless comparisons. |
uc(x) |
Converts x to UPPER CASE. Use for caseless comparisons. |
d2n(x) |
Converts a text string representing a date (expressed in one of the formats that Foswiki can parse) to a number of seconds since 1st Jan 1970. This is the format dates are stored in inside Foswiki, and you have to convert a string date using d2n before you can compare it with - for example - the date an attachment was uploaded. Times without a timezone are assumed to be in server local time. If the text string is not recognised as a valid date, then d2n will return undefined . |
NOT |
Invert the result of the subquery |
AND |
Combine two subqueries |
OR |
Combine two subqueries |
() |
Bracketed subquery |
The same operators are supported by the %IF and %QUERY macros.
You can get the current time for date comparisons using SpreadSheetPlugin, thus: %CALC{"$TIME()"}%
If you want to know if a field is undefined (has never been given a value) then you can compare it with undefined
(this requires that no field called undefined
exists in the form).
In the operators ( = != ~ =~ < > <= >= NOT AND OR
) an undefined operand is treated the same as numerical 0. For lc uc d2n
an undefined operand will give an undefined result. For length
and undefined operand will give a result of 0.
Putting it all together
When a query is applied to a topic, the goal is to reduce to a TRUE or FALSE value that indicates whether the topic matches that query or not. If the query returns TRUE, then the topic is included in the search results.
A query matches if the query returns one or more values when it is applied to the topic. So if I have a very simple query, such as
"attachments"
, then this will return TRUE for all topics that have one or more attachments. If I write
"attachments[size>1024 AND name ~ '*.gif']"
then it will return TRUE for all topics that have at least one attachment larger than 1024 bytes with a name ending in
.gif
.
Gotcha
- Remember that in the query language, topic names are constants. You cannot write
User.UserTopic/UserForm.firstName
because User.UserTopic
will be interpreted as a form field name. If you want to refer to topics you must enclose the topic name in quotes i.e. 'User.UserTopic'/UserForm.firstName
Examples
Query examples
-
attachments[name='purdey.gif']
- true if there is an attachment call purdey.gif
on the topic
-
(fields[name='Firstname'].value='Emma' OR fields[name='Firstname'].value='John') AND fields[name='Lastname'].value='Peel'
- true for 'Emma Peel' and 'John Peel' but not 'Robert Peel' or 'Emma Thompson'
-
(Firstname='Emma' OR Firstname='John') AND Lastname='Peel'
- shortcut form of the previous query
-
HistoryForm[name='Age'].value>2
- true if the topic has a HistoryForm
, and the form has a field called Age
with a value > 2
-
HistoryForm.Age > 2
- shortcut for the previous query
-
preferences[name='FaveColour' AND value='Tangerine']
- true if the topic has the given preference settings and value
-
Person/(ClothesForm[name='Headgear'].value ~ '*Bowler*' AND attachments[name~'*hat.gif' AND date < d2n('2007-01-01')])
- true if the form attached to the topic has a field called Person
that has a value that is the name of a topic, and that topic contains the form ClothesForm
, with a field called Headgear
, and the value of that field contains the string 'Bowler'
, and the topic also has at least one attachment that has a name matching *hat.gif
and a date before 1st Jan 2007. (Phew!)
-
length(fields[NOT lc(attributes)=~'h'])
- the number of fields that are not hidden
Search examples
Find all topics that are children of this topic in the current web
%SEARCH{"parent.name = '%TOPIC%'" web="%WEB%" type="query"}%
Find all topics that have an attachment called 'grunge.gif'
%SEARCH{"attachments[name='grunge.gif']" type="query"}%
Find all topics that have form
ColourForm
where the form field 'Shades' is 'green' or 'yellow' but not 'brown'
%SEARCH{"(lc(Shades)='green' OR lc(Shades)='yellow') AND NOT(lc(Shades) ~ 'brown')" type="query"}%
Find all topics that have PNG attachments that have been added since 26th March 2007
%SEARCH{"attachments[name ~ '*.png' AND date >= d2n('2007-03-26')]" type="query"}%
Find all topics that have a field 'Threat' set to 'Amber' and 'cold virus' somewhere in the topic text.
%SEARCH{"Threat='Amber' AND text ~ '*cold virus*'" type="query"}%
Find all topics newer than one week old
%SEARCH{"info.date >= %CALC{"$TIMEADD($TIME(), -7, day)"}%" type="query"}%
Related Topics: SearchHelp,
VarSEARCH,
FormattedSearch,
Foswiki:System/QuerySearchPatternCookbook