Data Cleaning Automation for Large Datasets
Data Cleaning Automation schedules batch runs of selected Flookup custom functions so you can process large datasets in Google Sheets. Jobs run on a schedule and write results to the output range you choose, which helps with large ranges that are difficult to handle in a single formula calculation.
On This Page
Open the Scheduler
In Google Sheets, open Extensions > Flookup Data Wrangler > Data Cleaning Automation. The sidebar lets you choose a function, define ranges and options, set a schedule, and monitor progress.
Supported Batch Jobs
Data Cleaning Automation currently schedules batch processing for five custom functions that are commonly used on large datasets:
| Job | What it does | Typical inputs | Output |
|---|---|---|---|
| NORMALIZE | Cleans and transforms values row by row, for example removing diacritics, keeping text or numbers, removing punctuation, trimming or changing case. For text and punctuation operations you can provide an optional stop-words/characters range. | Source range (single column), operation, optional stop range | One cleaned value per source row written to the output range |
| FUZZYSIM | Compares values in two ranges row by row and returns a similarity score. | Left range, right range, compare mode | One score per row written to the output range |
| FLOOKUP | Finds the best fuzzy match for each lookup value in a table and returns the matched value. You can also return the score or row position with the match. | Lookup range, table range, lookup column, return column, threshold, output mode | Matched value (and optional score or position) per lookup row |
| ULIST | Extracts unique rows from a source range based on a key column, using exact or fuzzy matching. | Source range, key column, threshold | Unique rows written to the output range |
| DEDUPE | Removes duplicate rows by key column, keeping the first or last occurrence, using exact or fuzzy matching. | Source range, key column, keep mode, threshold | Deduplicated rows written to the output range |
For formula-by-formula syntax and examples, see Custom Functions.
How Scheduling Works
- Scheduled runs. You can run jobs every 15 minutes, hourly or daily, with auto-chaining that saves progress and continues on the next run until the job completes.
- Output to ranges. Results are written to the output range you specify. Cells in the target area are overwritten by the job's results.
- Progress. Large jobs can take more than one run to finish. Progress is saved so work can continue on later runs.
- Status. The sidebar shows the job state, progress and any recorded errors.
Configure and Run a Job
- Select a function. Choose NORMALIZE, FUZZYSIM, FLOOKUP, ULIST or DEDUPE. The form shows the fields required for that job.
- Set ranges and options. Enter ranges in A1 notation (you can include a sheet name, for example
Sheet1!A2:AorData!A2:D), then set columns, thresholds or modes as needed. Choose the output range top-left cell where results should be written. - Set the schedule. Pick minutes, hours or days and the frequency (and start time for daily schedules).
- Save schedule. Click Save schedule to create or update the trigger and stored job configuration.
- Run now (optional). Click Run now to process a chunk immediately.
Manage Schedules
- Pause / Resume. Pause stops future trigger runs while keeping saved progress. Resume recreates the trigger and continues from the saved position.
- Clear. Stops the trigger and clears the saved schedule and progress for the selected job.
- Status. The sidebar shows state, progress, last run, next run and last error for the selected job.
Important Notes
- Very large jobs, especially fuzzy matching and deduplication jobs, can take multiple runs to complete.
- Make sure the sheet and ranges referenced by a saved job remain available. Renaming or deleting a source/output sheet can cause runs to fail until the job is updated.
- Custom function formulas in cells and scheduled batch runs are separate ways to run the same logic. Formulas calculate in the sheet, while scheduled runs process ranges in batches on a schedule.