Skip to content

Reschema and Data Masking

  • MongoDB

Reschema writes a collection again in another shape: fields moved or nested, converted to another type, left out, or computed from others. Data Masking writes a copy for people who must not see the data: characters hidden, values replaced or set to null, numbers and dates moved by a random amount. Both work from a rule per field. Querybara turns the rules into an aggregation pipeline, and the server runs it into another collection, so the documents never leave the server and the source collection stays as it is.

  • Choose Reschema… or Data Masking… from a collection’s menu in the explorer.
  • Run MongoDB: Reschema… or MongoDB: Data Masking… from the command palette.
  1. The panel reads a sample of the collection and lists its fields, one rule each, set to Keep. Read fields again reads a new sample.
  2. For each field, pick a Rule, and fill in Into and Settings where the rule has them (see below). The close button at the end of a row removes its rule, and the field stays as it is.
  3. In Reschema, Add computed field adds a field made by an aggregation expression.
  4. Tick Only the fields listed to keep only the fields that have a rule.
  5. Choose Preview. The first ten documents show under Before (the first documents) and After, without writing anything.
Rule Offered in What it does
Keep Both Keeps the value. A new path in Into moves it, or nests it: city into address.city.
Convert to Reschema Converts the value to string, int, long, double, decimal, bool, date or objectId. A value that cannot be converted stays as it was.
Leave out Both Drops the field.
Computed Reschema An aggregation expression in mongosh syntax, such as { $concat: ['$first', ' ', '$last'] }.
Mask characters Data Masking Shows the first and last characters you set and replaces the rest with the mask character (* to start with). Strings only; a short string is masked whole.
Replace with Data Masking A fixed value in mongosh syntax, such as 'hidden'. A value that starts with $ stays a value.
Set to null Data Masking Sets the field to null.
Randomise number Data Masking Moves a number by up to ± a percentage (10 to start with). Int32 and Int64 values stay whole numbers of their type, and Decimal128 stays Decimal128.
Randomise date Data Masking Moves a date by up to ± a number of days (30 to start with).

Each masking rule changes only values of its type; other values pass through unchanged.

Querybara refuses rules that cannot work and names the rule: two rules that write to the same place, a field moved inside itself, a field other than _id written to _id, a value or expression that does not parse, or a field left out while another rule writes into it.

Under the rules, the pipeline shows as mongosh text. Open in Aggregation Editor opens it in the aggregation editor.

  1. In Write into, name the collection to write: orders_reschema or orders_masked for orders, to start with. It must be another collection than the source.
  2. Pick how to write it: replacing what it holds ($out), which also creates the collection when it is missing, or merging by _id ($merge), which replaces the documents with the same _id and adds the rest.
  3. Choose Run. The confirmation shows the exact aggregate(); choose Run and write.

Run always asks first, on every connection. A read-only connection does not run it.

The random rules use $rand, which needs MongoDB 4.4.2 or later.

Documents Querybara 0.1.19 · built from89dc752