Release Notes

New Features

• Added new API method GetAgentExtendedInfoAsJson.
• Added HTTP Version property to the Agent command.

Changes

• The agent option JSON Parser Compatibility Mode is now always set to false by default. Existing agents relying on compatibility mode may need this option set to true specifically.

Bugs

• An error may occur when exporting data from multiple agents to the same table.

New Features

• Added action command option Capture Requests Regex.
• Added agent version to info returned by API method GetInfoForAllAgentsAsJson.

New Features

• Added Python scripting.
• Added ScriptUtils methods PostDataWithStatus and GetDataWithStatus.

Changes

• Export property Export Robots Files can now be set to None, Manual or Automatic.

Bugs

• API method GetAgentScheduleInfoAsJson returns invalid JSON.

New Features

• New action command option Separate Parser Proxies will create a new proxy loop for the action command and all sub-commands.
• The dynamic browser engine is no longer started for agents that don't use dynamic browsers, resulting in slightly better performance for such agents.
• Added API methods to get and set binary values.

Bugs

• Error running agents with SQL Server or MySQLas internal databases.

Bugs

• Content Grabber fails to load agents on new installations.
• Content Grabber always restores agents from a cloud repository to the default agent location.

New Features

• New Agent option Max Run Time can be used to stop and fail an agent that has run longer than the specified timespan.

Changes

• Post Status renamed to Web Notifications and can now be configured to notify on both agent start and completion.

Bugs

• Agent run time is not calculated correctly if an agent is stopped and later continued.

Bugs

• Form Field commands are unable to correctly fill form fields on some websites.

Bugs

• The Screenshot command takes incorrect screenshots when an agent is run by the scheduler or the API.

Bugs

An error may occur when creating a new cloud proxy pool.

Bugs

• An Access Denied error may occur when restoring an agent form the cloud.
• An error may occur when viewing logs if the Internal Database is set to MySQL.

Bugs

• Password required when using SFTP with private key.
• Error creating now configuration in cloud.

New Features

• SFTP and FTPS support added to data delivery.
• Remote directory option added to FTP data delivery.
• A configuration file can now be used to enforce specific agent settings for all agents in a directory.
• Proxy Pools added to Content Grabber Cloud.
• Agent Configuration added to Content Grabber Cloud.
• Method AddNotificationMessage added to most script arguments.
• New Agent option Load Start URLs can be set to false to prevent the Agent command from loading URLs.
• Ability to export robots.txt files encountered during a run.
• New command-line tool UpdateCg.exe can be used to upgrade an installed version of Content Grabber to the latest version or a specific version.

Bugs

• An error may occur when using an Exit If Data Exists command with MySQL as internal database.
• A warning is often logged incorrectly when an agent run completes.

New Features

• New scheduling option Windowless.

Bugs

• The retry count doesn't increase when using an Execute Script command to restart and retry an agent.

Changes

• Agents no longer log an error when restarting web browsers.

New Features

• New File menu Move or Rename in the Content Grabber Editor can be used to move or rename agents while retaining their unique agent ID.
• New Agent option Split Proxies can be used to dedicate a subset of proxies to individual performance sessions.

Bugs

• Input data is not spread evenly among Performance Sessions.

New Features

• Browser Tools added to Tools menu in the Content grabber editor.
• New option New On Proxy Rotation added to Profile Rotation and User Agent Rotation.
• More complete removal of cookies and local storage.

Changes

• Using DistributeData in a Data Export script now has now effect in debug mode.

Bugs

• Unable to clear the options Container Link and Dependant Container.

Bugs

• The Production Mode is always set to true when an agent is run from the Run screen.

Bugs

• The Production Mode button on the Run screen has no effect. The Application setting Disable Production Mode is always used.
• Using the DistributeData method in a Data Export script ignores the Production Mode setting and distributes data even when an agent has been manually stopped.
• Self-contained agents display text with references to menus that don't exist.

New Features

• New Application option Disable Production Mode can be used to disable Email Notifications, Post Status and data delivery by default.

Changes

• Email Notifications and Post Status is no longer executed when manually stopping an agent.

Bugs

• Retrying commands doesn't always work properly.

New Features

• Custom JavaScript editor.

Bugs

• A generic error is shown when the CSV reader fails.

New Features

• New Command Script return option RestartRetryAgent.

Bugs

• The option Rotate User Agent has no effect during design and debugging.
• Errors are not always cleared when successfully retrying errors.
• Storage and Cookies are not always cleared when opening an agent.

New Features

• New method CompleteProcessingRow on the DataRow class can be used to force the current processing data row to be saved and marked as completed.

New Features

• New option to randomize the size of the web browser.

Bugs

• The Capture command option Save in Parent may cause an error at runtime.
• Unable to set User Agent rotation.

New Features

• New error handling option Clear Retry Count on Success.
• New method ClearRetryCount on the DataRow class.

Changes

• Exit Or Retry commands now always clear the retry count on success if the command is configured to retry.

Bugs

• VB Script libraries don't function properly.

Bugs

• Missing dependency error in version 2.51.4.

Changes

• The ScriptUtils methods PostData and GetData are now called at design time.

Bugs

• Incorrect files were included in the 2.52.3 installer which caused the Trial Edition to request a serial key.
• The Download Page command sometimes save downloaded HTML pages with a PDF extension.

Changes

• The ScriptUtils methods PostData and GetData are no longer called at design time.

Bugs

• The ScriptUtils methods PostData and GetData hangs at design time.

New Features

• New script utils methods GetJsonStringValue and GetJsonStringValues.
• New Link command option Skip First Link.

Bugs

• The HtmlNode property is always set to null in Pagination transformation scripts.
• The Script Utils methods GetData and PostData hangs.

New Features

• New option to enable/disable canvas reading. Canvas reading is disabled by default.

Bugs

• Cookies and storage are not always deleted correctly.

Changes

• All popup windows are now redirected to standard browser windows for better performance. The browser target must now be set to Popup for websites that depends on a real popup window that can be controlled from the parent browser window.

Bugs

• The Delay property for Date Range data providers is always set to days.

Bugs

• Download Document commands cause an error in some situations when Export URL is set to false.

Bugs

• Download Document commands don't export the correct URL when using Click to Download
• A URL is not exported when using File Capture commands that convert the extracted file without exporting the file.

New Features

• The Date Range data provider now supports date ranges from a specified date to a date in the past.

Changes

• HTML Help file no longer included in the installer. Content Grabber will link to online help instead. HTML Help files will be available as a separate download.

Bugs

• Sorting and filtering of schedules in the scheduler window does not work correctly. 
• Content Grabber crashes when executing a Download Document command configured with Click to Download if the click triggers a new window to be opened.
• An incorrect value is saved as Delayed Days when configuring a Date Range data provider.

Bugs

• Deleting an agent does not always delete its entry in the Application Database.
• Selecting a document link and then choosing to capture the document URL, incorrectly creates a Download Document command instead of a Web Element Command.

Bugs

• Imported agents get a new ID if they don't already exists when imported.

New Features

• Agent diff tool available when connected to an online agent repository.

Bugs

• The internal scheduler does not always work correctly.

Bugs

• Error getting latest version from Agent Repository if the agent doesn't already exist locally.

Bugs

• Exit If Data Exist commands don't always work, and may prevent agents from running.

New Features

• The Exit If Data Exist command can now use an external database table to store hash keys. The table can be shared by multiple agents.
• Dynamic browser upgraded to latest Chromium version.

Changes

• Parser Cache Method set to Never by default.

Bugs

• Error executing action commands that use a parser and don't target a new browser.
• Transform Page commands don't work when using a parser.
• Retry error handling for URL commands that iterates through multiple URLs, will exit the URL command on failure, instead of continuing with the next URL.

Bugs

The XPath editor in the Content Grabber Editor's status bar is not working correctly and has been removed.

New Features

• Dynamic browsers can now load WebGL.
• Flash can now be turned on for individual agents at runtime.

Changes

• Manually refreshing the current document in the Content Grabber editor will now refresh all frames.

Bugs

• Agent commands are not multi-threaded when using dynamic browsers.
• An error occurs when manually refreshing the current document in the Content Grabber editor.
• The error "The agent took too long to start" may occur incorrectly when starting an agent on some systems.
• An error occurs when a Capture command is configured to index captured data and SQLite is used as internal database.
• Unable to stop an agent from the Content Grabber Scheduler window if the agent is running multiple performance sessions.
• An error occurs when starting an agent from the Content Grabber Scheduler window if the agent has been built using the Content Grabber Professional edition.

New Features

• A separate XML description file can now be exported together with an agent. The description file contains a plain text view of all agent settings.
• Capture command options Always set design value and Make Data Available to Parent Commands can now be set from the Agent Explorer context menu.

Changes

• Capture command option Copy to Parent has been renamed to Make Data Available to Parent Commands and now copies the captured data value to all parent data rows, no matter if there's a parent command matching the Capture command name. If there's a matching parent Capture command, the data captured by the parent Capture command will be overwritten. If there's no matching parent command, the data will be available to parent commands like any other captured data, but the data will only be kept in memory and will not be exported.

Bugs

• An error may occur when switching between Navigation and Selection mode in the Content Grabber editor.
• Exit or Retry command options Retry Delay and Rotate Proxy have no effect.

New Features

• New API method UpdateOrAddSchedulesForAllAgents can be used to add or update a schedule for all agents in a directory.
• New API method DeleteSchedulesForAllAgents can be used to delete all schedules for all agents in a directory.

Bugs

• API method GetInfoForAllAgentsAsJson does not accept an access key parameter.
• Error adding API access keys.
• Schedules with a run internal of more than 24 days don't work correctly.
• An error occurs when editing an agent after deleting the internal database.
• The Schedule description is incorrect for schedules with a basic internal type of days.

Bugs

• An error occurs when editing the schedule of an agent that has not been copied to a server, but has not been imported or saved on the server.
• Content Grabber crashes when adding a Calculated Value Calculate HASH command from the command configuration panel.
• Remove Duplicate and Exit If Data Exist commands don't show all available Capture commands when added from the command configuration panel.

New Features

• New Capture command property Copy to Parent copies the extracted data to the first parent capture command with the same name if it doesn't already contain data.
• New option Page Not Found Handling can be used to specify what action to take when a web server returns a Page Not Found error. 

Bugs

• The Content Grabber editor may crash when viewing the Nodes panel.
• Content loaded using the json:// and html:// protocols is encoded incorrectly.

Bugs

• Retrying Link and URL commands may cause an infinite loop that cannot be stopped.

New Features

• New command Exit If Data Exist can be used to exit a specified container command when encountering data that has previously been extracted. 
• New command Compute Hash can be used to compute and save hashes of extracted data. 

Bugs

• Loading HTML or JSON using the html:// or json:// protocols don't work at design time.
• Error using a Wait for Content command just after opening a new dynamic browser.
• No debug data is exported when using the Old Data option Keep Some and Don't Export.
• The anchor attributes Position and Node Position don't work.
• 32bit runtime package does not work on 64bit Windows.

Bugs

• Loading HTML or JSON using the html:// or json:// protocols don't work at design time.

New Features

• The performance of the JSON parser has been improved. Existing agents using the old JSON parser most be updated and the agent property "JSON Parser Compatibility Mode" set to false, to take advantage of the improved performance.
• A warning is now generated at design time when optional action activities time out.

Bugs

• When double clicking on an image, there's no option to capture image URL, but only an option to capture URL which is always empty.
• Highlighting in the editor may not work correctly on websites using zoom and transform styles.
• Error at design time when using certain filters and anchors.
• An error message is not always shown at design time when failing to download a document.
• Error executing a Download Document command at design time if the agent has not been saved after the command was added.

Bugs

• Some Content Grabber system files are not signed correctly.
• Basic Content Grabber schedules are converted into CRON expressions, but CRON expressions are not capable of defining equivalent schedules.

Bugs

• Email services attempt to authenticate no matter if username and password is left empty.

New Features

• New option JavaScript Parser Timeout can be used to increase parser timeout if running many concurrent agents on a slow computer.
• Small internal performance improvements.

Bugs

• An agent that uses multiple concurrent web browsers fail when executing a action command asynchronously if the command is not configured to open a new web browser. 

Bugs

• Installer window freezes while optimizing Content Grabber performance.

Bugs

• Screenshot commands don't work correctly when a webpage changes the web page on scroll.

New Features

• An agent can now be configured to clear both cookies and local storage.
• Slightly faster startup of the Content Grabber Editor.

Changes

• The Clear Cookies Tools menu has been renamed to Clear Cookies & Storage and will now clear both cookies and storage.
• Action command options Clear Cookies and Clear Session has been replaced with the option Clear Cookies & Storage.

Bugs

• Default and Direct proxy settings don't work correctly when using a HTML Parser.
• Scheduled agents don't run if they are created with a Professional edition of Content Grabber.

New Features

• New Agent property to set browser language.

Bugs

• Parser cache mode is not applied when downloading images.
• Accept-Language header is not set when using a HTML Parser.

New Features

• Session ID column added to Content Grabber Scheduler overview window.

Bugs

• Session ID not populated correctly when editing a schedule in the Content Grabber Scheduler.
• The Content Grabber scheduler fails to start in non-English versions of Windows.
• White spaces in email addresses cause an error.

Bugs

• HTML Parser multi-threading does not work correctly if number of web browsers is set to 1.

Changes

• Help file updates.

New Features

• Event monitoring can not be turned off for an agent.

Bugs

• When copying a HTML Capture command, the properties Concatenate Multiple Web Elements and Concatenate Content Separator are not copied.
• AJAX calls returning JSON objects are not handled correctly at design time.

New Features

• New API functions to add, update, delete and run schedules.
• API access keys can now be assigned rights to run agents, access agent data, schedule agents, or full access.

Bugs

• API access keys don't always work correctly.

New Features

• Summary panel added to View Data windows.
• New function SetProxy added to Execute Script commands. This function forces an agent to use a single specific proxy.

Bugs

• Settings proxies in an Execute Script command has no effect when debugging an agent.

Bugs

• An error may occur when opening the Content Grabber Scheduler window.

Bugs

• Content Grabber service fails to start properly.

New Features

• New scheduler with support for agent sessionIDs, agent input parameters, and Cron expressions. Multiple schedules can be added for each agent, and agent schedules can be exported/imported together with the agent. Windows credentials are no longer required when adding a schedule to an agent.

Changes

• A new Content Grabber scheduler replaces the old scheduler that relied on Windows Task Scheduler. The old scheduler will continue to be available and supported. Options related to the old scheduler are available from the Tools menu in the Deprecated options group.
• The Content Grabber editor will now load a maximum of 1000 items in a JSON array. This will help improve performance at design time when handling large JSON files. All data will still be extracted at runtime.

Bugs

• Unable to edit Content Transformation scripts on data consumer commands if the data provider is not available.
• Backslash characters are not escaped automatically in input parameters. 
• An agent will not stop if it's stopped quickly before the initialization script completes.
• The design time value for Download Document commands is not always set correctly.
• Change date is exported as insert date when change tracking is turned on, and the agent exports to a file format.
• The Content Grabber editor fails to load a page properly when a website makes AJAX calls of type JSON and invalid JSON is returned.
• Agent properties set in an initialization script are ignored.

New Features

• Data distribution now supports Amazon S3 Buckets.

Bugs

• Content Grabber crashes on startup if an invalid gateway proxy is enabled.
• Logging does not work in self-contained agents.

New Features

• Email delivery and notifications now support implicit SSL on port 465.
• API access can now be limited to certain groups of agents by using access keys.
• New agent option to obey robots.txt rules. A page warning is generated by default in the Content Grabber editor if a page load is disallowed by the robots.txt file. 

Changes

• Embedded browser updated to Chromium 59.0.3071.109.

Bugs

• Screenshot commands don't work at runtime.

Changes

• Drop Box distribution updated to Drop Box API v2.
• Drop Box distribution now always overwrites files.
• Google Drive distribution now always overwrites files.

Bugs

• URL commands that use the custom protocols json:// or http:// fail when debugging.
• Content Grabber does not prevent naming commands with disallowed characters.
• Custom Regex function aggregate does not work.
• Drop Box distribution does not work because it relies on Drop Box API v1 which is no longer supported.

Bugs

• Unable to add multiple input values in self-contained agents.

Bugs

• API methods GetAgentDataAsXslZip and GetAgentDataAsCsvZip don't always work correctly.

Bugs

• Pagination commands don't work correctly unless a fixed pagination limit is set.

New Features

• Additional command info added to Branch and Wait for Content commands.
• Minor GUI enhancements.

Bugs

• Unable to clear export options "Export Rows to Columns Value Command" and "Export Rows to Columns Name Command".
• Pagination commands repeat page 1 when pagination limit is set to 1. 
• Content Grabber may crash when changing input data source.

Bugs

• Retry count has no effect when a custom script restarts and continue an agent.

New Features

• New Data added to Search data set option for Remove Duplicate commands.
• Improved agent startup time. An agent must be saved before this performance improvement takes effect.

Bugs

• Agents with many proxies are slow to load, and may even cause agents to fail to run because of startup timeout.

Bugs

• Unable to change Search data set for Remove Duplicate commands.
• windowscrolls and scrolls functions stop scrolling if a scroll doesn't result in an AJAX call.

Bugs

• Screenshot Log commands are not documented in the manual.
• Content Grabber may crash when opening an agent that uses a proxy list if there are no proxies in the list.
• Content Grabber hangs when scrolling to the end of a web page.

New Features

• Added events windowscrolls and scrolls which will do multiple scrolls faster than repeated scroll actions.
• Added option Auto vacuum to SQLite internal database configuration.

Changes

• An action command with Wait for Content sub-commands, that has Wait for Content set to None, will not wait for content, but will complete the action once the content appears in the web browser. The command will also complete the action if the normal wait conditions are meet, even if the wait content hasn't yet appeared in the browser (or never appears in the browser).

Bugs

• Wait for Content commands select web elements in the Content Grabber editor as if their XPaths were not restricted to strict XPath version 1.0.

Bugs

• Content Grabber fails to parse pages with IFrames.

New Features

• Cache Method added to agent properties. This option can be used to ensure parsers never retrieve pages from cache.

Bugs

• JSON properties with null values are ignored by the JSON Parser.
• Content Grabber does not block WebRTC leaks, which can allow JavaScript to retrieve a computer's IP address even when using proxies.

Bugs

• Screenshot log message is displayed every time debugging completes.

New Features

• Added button to open exported Excel spreadsheet to the Run screen.
• Added New Line Encoding to CSV export.

Bugs

• File name transformation does not work for Download Page commands.

Bugs

• Changing a Condition Script doesn't trigger an indication that the agent has changed.

Bugs

• The form field input value is not passed to exec events when executing a Form Field command.

Bugs

• CSV file export now uses the .txt file extension by default when values are delimited by tab.
• The file encoding for CSV export causes problems when BOM is not included in UTF-8.

New Features

• Content Grabber editor performance improvements. 
• An agent can now be configured to not export data.

Bugs

• The file encoding for CSV export is set to UTF-8 BOM by default which causes problems in some applications. The encoding should be changed to UTF-8 without BOM.

Bug Fixes

• Incorrect icon for Download Image commands.
• The Referer request header is set incorrectly.

Bug Fixes

• The transformed capture panel on most configuration screens cannot be resized.

New Features

• A description can now be added to commands, which will be displayed on the command info panel.

Bug Fixes

• Command transformation scripts fail when used in a custom script library.
• Unable to set a page load percentage as a success criteria.

New Features

• Agent loading screen added to self-contained agents.
• Minor performance enhancements.

Changes

• The message displayed when saving data export settings in self-contained agents has changed to make it clear the agent needs to be rerun to export data in a new format.

Bug Fixes

• The start button in self-contained agents doesn't become inactive immediately after pressing it.

New Features

• Add Sub-Command and Edit buttons added to command info panel.

Bug Fixes

• Unable to navigate files on the local file system, because file URIs are calculated incorrectly.

New Features

• Export Row Count now available in Post Status.

Changes

• Agent multitreading options have changed to \"Max Active Browsers\" and \"Max Active Parsers\", and now more accurately controls the number of active parsers and dynamic browsers.

Bug Fixes

• Email notifications include the wrong value for Export Row Count.

Bug Fixes

• The User Agent is not always set correctly for HTML Parsers at design time.

Bug Fixes

• The manual is missing a topic about the Post Status functionality.
• Custom Content Grabber Regex operations fail if they are trailed or preceded by spaces. 
• The default script library C# project has the Condition Script unit test in a wrong name space.

Bug Fixes

• Post Status does not work correctly at runtime. 

Bug Fixes

• Condition script is missing from the default script library.
• The HTML Parser uses a different User Agent than the dynamic browser by default.
• Unable to load a specific URL into a browser tab in some situations.
• Incorrect scaling on many screens in the Content Grabber editor when using high DPI settings.
• Post Status does not always work correctly at runtime. 

New Features

• Added Post Status screen.

Bug Fixes

• HTML Parser User Agent is not set to the same user agent as the web browser by default, which causes issues on some websites when switching from dynamic browser to HTML Parser.

New Features

Added Notify on successful data extraction and Success Email Addresses to the notification screen.

Bug Fixes

• Some agents export duplicate data when they shouldn't.

Bug Fixes

• Some websites fail to make AJAX requests in the Content Grabber editor, which may cause the website to load incorrectly.

Bug Fixes

• Continue & Retry Errors does not work correctly on the screen that shows all performance sessions.
• Setting the Default values check box when editing a URL command does not disable data transformation.

New Features

• Improved reliability and performance when multi-threading.

Bug Fixes

• Page load success criteria don't work correctly.
• An error may occur when executing a Form Field command that doesn't execute an action.
• Potential thread handle leak when multi-threading. This has the potential to cause stability issues when executing many high performance agents on a single server.

Bug Fixes

The Continue & Retry All button on the screen that shows all performance sessions, will appempt to start sessions that are complete and have no errors.

New Features

• Performance session progress added to screen that shows all performance sessions.
• Added Continue & Retry All button to screen that shows all performance sessions.
• Added Start All button to screen that shows all performance sessions.
• Added Stop All button to screen that shows all performance sessions.

New Features

• Added unescape_string method to Regex transformations.

Bug Fixes

• The JSON Transformation Script Editor does not use the raw page content as input data, which can make it difficult to generate a correct transformation.
• Unable to overwrite an existing agent in some situations.

Bug Fixes

• The script property DatabaseConnection is not available in Data Export scripts.
• The method WriteDebug does not work correctly in Raw Data Export scripts.

Bug Fixes

• Changing debug speed during debugging doesn't take emmediate effect.

Bug Fixes

• The script library panel is expanded by default on the Script Editor screen.

New Features

• New property HasDoneAction available in most scripts.

New Features

• New methods in the RuntimeData Script Utilities class.

Bug Fixes

• An error occurs when trying to import an agent or upgrade Content Grabber if the computer's TEMP or TMP environment variables are configured incorrectly, or points to paths that are inaccessible.

Bug Fixes

• Unable to upgrade templates.
• Content Grabber icons are not associated with Content Grabber files.

New Features

• Command and script templates can now be referenced instead of copied into an agent.
• Added suggested command types to the New Command drop-down panel.
• Added Search button to the View Data screen.

Changes

• Templates created with versions lower than 2.33.0 are no longer available, but can be upgraded to version 2.33.0+ from the Tools menu.
• Templates are now stored in individual files instead of one big file. This makes it easier to copy or move templates between computers.
• Some menu items have been moved from Application Settings to Tools.

Bug Fixes

• Retrying commands doesn't always work as intended.
• Logging does not work in self-contained agents.
• Input data is always enclosed in double quotes in self-contained agents, even when it's not required.
• Some web forms that shoud be able to be processed with a HTML Parser cannot.

Bug Fixes

• Error downloading files from direct HTML pages.

Bug Fixes

• Content Grabber crashes when pagination limit is set to one page.
• The Form Field option Set Form Field Value has no effect. This causes checkboxes to be set incorrectly most of the time.
• The action option Scroll to end of page does not always work correctly.

Bug Fixes

• The Content Grabber editor crashes when trying to open a link in a new browser if that link does not start an action and the browser does not already exist.
• A page load error is not reported when an action command configured to open a new web browser fails to open a new web browser.
• Unable to enable Min. page loads on Agent success criteria screen.
• Unable to change default JSON export settings.
• The Wait for Content timeout value keeps increasing automatically every time an agent is loaded.

Bug Fixes

• Error running an agent using MySQL as internal database.
• A database lock error may occur when running multiple sessions.

Bug Fixes

• Switching from Dynamic browser to HTML Parser can make it impossible to configure an action command to load a page in a new browser.
• Incorrect status information while running an agent.

New Features

• Performance optimizations.

Bug Fixes

• Content Grabber is unable to handle websites that make exact copies of web elements. This is the case for certain areas of amazon.com.
• Error retrying Pagination commands.

New Features

• New agent option Delay Between Performance Sessions can be used to specify a delay between session startups when starting multiple performance sessions at once.

Bug Fixes

• The RuntimeData class is not documented in the manual.

New Features

• Performance improvements when using SQLite or SQL Server as internal database.

New Features

• Performance improvements.

New Features

• Performance improvements when adding many data rows per loaded page.

Changes

• It's no longer required to execute a Web Element List to be able to select web elements inside the web element list.

Bug Fixes

• No data is processed when a performance session specifies a data set larger than the total number of data entries.

Bug Fixes

• The action timeout value File Download Started has no effect and a default value that is often too low is used.

Bug Fixes

• An agent may exit prematurely after retrying a Link command.
• An error may occur when Continuing an agent.

Bug Fixes

• The Agent option Keep Connection Alive has not effect.

New Features

• Added Use Test Set to the Agent Explorer context menu.

Bug Fixes

• Error handling options on Download Document commands have no effect.

New Features

• Added Test Sets to data provider configuration, which makes it possible to run tests on a limited data set without having to change data source.

Bug Fixes

• Proxy API configuration doesn't always work correctly.
• Some scripts are not executed correctly at design time when using the option Always Update Design Value.

Bug Fixes

• Status does not switch to the first session when starting all performance sessions from the Run screen.

New Features

• New option Always Separate Session Export Data can be used to ensure all session data is always exported separately.
• New commandline and API parameters agent_import_path and agent_import_name can be used to control to where an agent is imported when running an exported agent.

Changes

• Performance sessions now always log to different log files.
• The Content Grabber runtime will now always log information to the agent's log folder or the specified log path. The runtime will no longer create default folders in the current user's document folder.

Bug Fixes

• The default Data Export script has an error.
• The Session ID available in data export scripts is always null when executing Data Export scripts from the Content Grabber editor.
• An error occurs when downloading a file if the web server does not provide a suggested filename.

New Features

The Session ID is now available to all scripts.

Action error handling now has an option to rotate, and optionally remove, the current proxy.

Exit or Retry commands now have an option to rotate, and optionally remove, the current proxy on retry.

Exit or Retry commands now have an option to delay retrying.

New Features

• New option Log error and exit on failure on Exit or Retry commands

Changes

• The option Delete on exit on Exit or Retry commands can now be used when both retrying and exiting.
• A column retry_count_ has been added to internal data tables. This means that some internal tables will need to be re-created, and agents that rely on internal data will be reset. To avoid the re-creation of the internal data tables, a column retry_count_ of type int can be manually added to the tables.

Bug Fixes

• Retrying list commands does not work correctly.

Bug Fixes

• An error may occur when debugging an agent with an Exit Or Retry command.
• The Retry Count property on Exit Or Retry commands are not saved properly.

New Features

• Added MPP proxy API support.

Bug Fixes

• Error testing proxy APIs.

Bug Fixes

• The Content Grabber editor hangs or crashes if an application proxy error occurs at startup.

New Features

• Added runMethod option to the simple Web requests.

Bug Fixes

• An error may occur when resuming agents using the API.

Bug Fixes

• An error may occur during file download.

New Features

• Server wide default paths can now be specified.
• When importing an agent that already exists, a new name can be specified for the imported agent.
• Multiple agents can now be exported at once.
• Content Grabber browser updated to new version of Chromium.

Changes

• The Block URLs option has been moved to agent settings, and is now applied agent wide. Also, multiple regular expressions separated by line break can be specified.

Bug Fixes

• Some websites that change page URLs without loading a new page are processed incorrectly.
• Converting a Visual Web Ripper project may crash Content Grabber.

New Features

• Logs can now be filtered by log level.
• Log level is now added to file logs.

Changes

• File logs are now tab delimited, so they can be loaded into Excel or other programs that can handle tab delimited files.

New Features

• All agent sessions or a specified range of agent sessions can now be started, viewed or deleted from the Run screen.
• Added support for Proxy APIs.

Changes

• The 3rd party proxy provider Proxicity has been removed. Proxy API with Proxicity as provider can be used instead.

Bug Fixes

• Insufficient log information when an agent fails to start.
• Web page redirect sometimes causes an error when using the HTML Parser.
• All possible agent exit codes are not listed in the manual.

New Features

• New method SetPageLoadError is available to most scripts.
• New Selection Missing option Log Page Load Error and Ignore Command.

Bug Fixes

• Command names containing single quotes sometimes causes an error.

Bug Fixes

• The Page Load Completed timeout has no effect on page loads in a HTML Parser at design time.
• Page load errors in the HTML Parser are sometimes lacking details about the error.

Bug Fixes

• The action option Block URLs has no effect.

Changes

• The export option Export Empty Row If No Data is now set to true by default.

Bug Fixes

• Changing options on a Wait for Content command does not always activate the Save quick bar menu.
• Some large web pages are not loaded correctly in a HTML Parser at design time.

New Features

• An Execute Script command can now remove the current proxy if the agent uses a proxy list.
• Commands can now be copied/pasted between agents in different Content Grabber editors.
• Multiple commands can now be added at once by specifying multiple command names separated by comma. Command names separated by new lines or tabs will be automatically converted to comma separated names when pasted into a command name field.

Bug Fixes

• Pagination commands fails if a command inside the pagination loop refreshes the page containing the pagination link(s).

Bug Fixes

• Content Grabber fails to download an image if a target web server suggests an invalid file.
• An incorrect warning is displayed every time a command is moved within the same browser sub-tab.

Bug Fixes

• Agents using performance sessions don't export any data.

Bug Fixes

• Content Grabber is unable to detect some methods in a default script library.

Bug Fixes

• Unable to scroll a web page in the Content Grabber editor by dragging the scroll bars.
• Web pages that change the page URL without loading a new page are not always processed correctly.

Changes

• Manually removing internal data now also removes all agent status information.

Bug Fixes

• Performance sessions with IDs containing odd numbers, or processing a data list with an odd number of entries, fail to loop through the entire data list.

Changes

• An agent exporting data to files and using performance sessions will now always export data from all sessions into a single file. An agent session will only export data if no other sessions of the same agent are currently running.

Changes

• The info panel on the Command Properties configuration window is now sized more appropriately.

Bug Fixes

• Extracted data is sometimes exported incorrectly to Excel and CSV when converting rows to columns

Bug Fixes

• The Content Grabber editor sometimes fail to track DOM events.
• Inline DOM events on delay loaded content are never tracked.
• The action event click() does not fire mouseenter and mousemove DOM events.

Bug Fixes

Navigate in Browser toolbar button does not work correctly in the Content Grabber editor.

Bug Fixes

• Content Grabber may crash when canceling a file download in the editor.

Bug Fixes

• Old browser processes are not closed properly when opening a new agent.
• Restoring an agent backup does not work properly.

Bug Fixes

• Resolve CAPTCHA templates generate incorrect commands.

New Features

• Most scripts now have access to a new property RuntimeData which provides access to internal data while an agent is running.

Bug Fixes

• Simple web requests to the Content Grabber service don't work.
• Extracted boolean values are sometimes handled differently when designing and running an agent.

Bug Fixes

• Custom assemblies are not loaded correctly when used in scripts.

Bug Fixes

• Some pages are loaded incorrectly when using a JSON, HTML or XML parser.

Bug Fixes

• Some JSON content is not parsed correctly by the JSON parser.

Bug Fixes

• The Content Grabber web browser does not work on some Windows configurations.

New Features

• Added default logging options for agents.
• A script library can now be any .NET assembly.
• A custom parameter can not be passed to methods in a script library.
• New script parameter IsParentCommandMissingSelectionOrData. This parameter is set to true if the parent container's selection was not found, or is iterating through an empty data list.

Changes

• When an agent is run from the commandline or API, the agent now logs to files by default.

Bug Fixes

• Content Grabber is unable to process local files.
• The export option Export empty row if no data does not work correctly.
• The Content Grabber web browser fails to load pages on some computers.

New Features

• Added support for browser zooming in the Content Grabber editor.

Bug Fixes

• Help file updates.

Bug Fixes

• Content Grabber may cause a JavaScript error on some websites.
• Content Grabber sub-processes may not shut down properly if Content Grabber crashes or if it's process is terminated.

Bug Fixes

• An error may occur when loading a URL with the HTML Parser.

New Features

• Added option to delete website cookies in the editor.

Bug Fixes

• Cookies are not always set correctly in the HTML Parser.

Bug Fixes

• The ImportSilent method may import an agent incorrectly.

Bug Fixes

• Remove Duplicate commands don't work properly on DateTime data types.
• The ImportSilent method may import an agent incorrectly.

New Features

• Content Grabber can now start multiple agent instances at once if the agent supports performance sessions. 

Bug Fixes

• An agent will try to process multiple list commands in sessions if more than on command has the option "Process in Session" set to true.

Bug Fixes

• Content Grabber sometimes mark visible web elements as if they were hidden.

New Features

• Added Performance Sessions which makes it easier to get multiple instances of the same agent to work on the same task.

New Features

• Added action event key that takes a character as parameter.

Changes

• Old action event key has been renamed keycode.

Bug Fixes

• Unable to to clear the Pagination command option Limit page count with fixed value.

New Features

• Added agent option to not load style sheets. When using a HTML Parser all inline styles will be removed as well.

Bug Fixes

• Some valid JSON content cannot be loaded in the JSON Parser.

New Features

• New override of API method ImportSilent makes it possible to import an agent to a specified location with a specified name.
• New API method DeleteAgent.
• New CSV export option Export Keys.

Bug Fixes

• Some buggy websites, that load more content on scroll, make it possible to indefinitely scroll when a browser window is at a certain height. This can cause an agent to run indefinitely.

New Features

• Added Google Drive distribution option.
• Added DropBox distribution option.
• Added JSON export target.
• Added options Has Headers and Ignore Parse Errors to CSV data providers.

Bug Fixes

• Incorrect default document converter script.
• Selections are not always marked correctly in the editor browser.
• Unable to create new shared script libraries from an agent that is not located in the default agents folder.
• Downloaded files are not distributed when exporting to Excel 2007+.
• Empty columns are inserted when embedding images in Excel 2007+.

New Features

• New API setting AgentStartTimeout can be used to specify the number of seconds an agent is allowed to start processing before a timeout occurs.
• Additional logging when an agent fails to start.

New Features

• New setting RuntimePath can be used in the API to specify the location of the Content Grabber runtime. If this setting is not specified, the runtime files must be copied to the same folder as the host application executable.

Changes

• The Create Website Images button has been moved to the Tools menu.

New Features

• New feature Create Website Images can be used to take screenshots of all types of web pages an agent visits.
• A warning message is now displayed when viewing data from an agent that converts rows into columns, since this functionality is only applied when exporting to file formats, and is not applied when viewing extracted data.

Changes

• The Page to PDF command type has been renamed to Download Page and can now download a page as PDF or HTML.

Bug Fixes

• Remove Duplicate commands don't work when debugging.
• Screenshot commands don't work properly on right-to-left language websites.
• Screenshot commands don't always work properly when Size of text, apps and other items is set to more than 100% in Windows.
• Agents using the obsolete database export option Add To Existing Data may cause an error when exporting.
• Agents with command names containing some special characters may cause an error.

Bug Fixes

• Documentation for Wait for Content commands is missing in the manual.
• The Content Grabber editor may crash on some script syntax errors.

New Features

• New command type Wait for Content can be used to force the parent action command to wait for specific URLs or web content.

Changes

• The Action properties Wait For Regex and Wait For Urls have been combined into a single property Wait For Content.
• The Action properties Wait For Regex Timeout and Wait For Urls Timeout have been combined into a single property Wait For Content Timeout.

Bug Fixes

• JSON content returned by a web server is not recognized as JSON if the response Content Type is set to application/javascript which is used by many web servers.
• Base meta tags are not always processed correctly when using a HTML Parser.

Changes

• Session cookies are no longer removed on agent start unless specifically configured in the agent command.

Bug Fixes

• Content Grabber crashes when trying to import a Visual Web Ripper project created with a recent version of Visual Web Ripper.
• The Delete button in the File menu does not work.

New Features

• Values from Local Storage and Session Storage are now available using the Page Attribute command.

Bug Fixes

• Content Grabber waits too long on some websites causing agents to run slower than they should.

Bug Fixes

• Custom HTML tags are not handles correctly.

Bug Fixes

• The option to limit pagination page loads with a fixed value is not saved correctly.

New Features

• New option to limit pagination page loads with a fixed value.

Bug Fixes

• Crash when using a HTML Parser and setting a form field in design or debug mode.
• Crash when setting an Exit or Retry command to use a specific command and then not specifying a command.

Bug Fixes

• Error exporting data to file formats when converting rows to columns and the converted list command generates no rows.

Changes

The Wait for XPath option now allows page loads while waiting for an XPath selection to exist.

Bug Fixes

• Error executing Page Transformation commands in design and debug mode when loading content directly into HTML parsers.

Bug Fixes

• Error exporting historical data to MySQL.

Bug Fixes

• HTML Parser requests never times out, no matter the configured time out value. This is a serious issue, especially when processing a large number of different websites, because some websites never return a response, and Content Grabber will then hang until stopped manually.

Bug Fixes

• Number of threads used when running an agent is not always correct when using HTML Parsers.
• It's not possible to see if a script is turned on or off on all command configurations screens.

Bug Fixes

• An error may occur at design time when converting a document that opens in a new window.
• "base" HTML tags without href attributes cause HTML parsers to generate wrong URLs.
• Database connection details saved in an agent maybe duplicated.

Bug Fixes

• An error may occur when a web page refreshes in a dynamic browser.

New Features

• New option to automatically add a capture command when creating a new Form Field command.

Bug Fixes

• Agent may fail to export when change tracking is turned on.

New Features

• CSV file content can now be edited directly from the Data Provider configuration window.

Bug Fixes

• Unable to cancel anchor selection mode.

Bug Fixes

• Exporting data since last successful run when using change trackikng does not work correctly.
• An error occurs when choosing captured data as data provider.

New Features

• New action option Ignore error codes can be set to false to force an error when a website returns an error code, even when the website returns content. Content Grabber ignores status codes by default when content is returned, because some websites return incorrect error codes, or return error codes that have special meaning and should be handled by the Content Grabber agent.

Changes

• HTML parsers no longer fail by default when an error code is returned from the web server, unless the option Ignore error codes is set to false. This makes the behavior the same as when using a dynamic browser. WARNING: this may cause existing agents to behave differently unless you set the option "Ignore error codes" to false.

Bug Fixes

• The ScriptUtils function GetData() makes a POST request instead of a GET request.

New Features

• New function insert_data is now available for Content Transformation regex scripts.

Bug Fixes

• When certain errors occur in Content Grabber and an agent process is restarted, web browser processes may be left running and never shut down.

Bug Fixes

• Page to PDF commands may fail depending on timing.

New Features

• New option Deliver data files can be turned off when distributing data, so only extracted files are distributed.
• New option Always update design value can be used on capture commands to make sure their design values are automatically updated when a new web page is loaded.
• When posting data with a URL, the pattern [CG_FILE_UPLOAD_PATH:...] can be used instead of the actual file content when upload a file.

Bug Fixes

• The option Deliver Extracted Files has no effect when delivering extracted data to email or FTP.
• Missing files may cause data export to fail when delivering data to email or FTP.
• The options Default Email Settings and Default Email Addresses are not saved when saving the email delivery settings.
• An agent may hang on websites that continuously make AJAX calls. 

New Features 

• Request method can now be specified when loading a web request.
• Post, header and method parameters can now be specified in any order when loading a web request.

New Features 

• Additional information added to the web request editor to make it easier to create file upload requests.
• Log settings can now be set in an Initialization script, but overriding the commandline parameters in the GlobalData object.

New Features 

• Response headers are not available in scripting.
• Custom headers can now be set for all requests in web browsers.

Changes

• The property LastBrowserMessage has been replaced with LastErrorResponse in scripting.

Bug Fixes

• Screenshot commands don't work.
• Error setting proxy to Fiddler.
• The base meta tag is not taken into consideration when extracting URLs.

New Features 

• Proxicity proxy support.

Bug Fixes

• Luminati proxy configuration is disabled.

New Features 

• Default email settings shared by all agents on a computer.
• Messages can be sent from an agent to its hosting application when using the API.

Bug Fixes

• Outdated help file topics.

New Features

• Support for historical data when using change tracking.
• New capture command option Exclude from change tracking.
• New command type Page to PDF saves a web page as a PDF file.

Bug Fixes

• Refresh Document commands cause the debugger to hang.
• Content Grabber may crash when editing commands that consume data.
• The Max. repeats option for link commands does not work.
• The Close Browser command doesn't properly close a browser.

Bug Fixes

• JavaScript on a target website is sometimes able to close an agent browser, which can cause the agent to crash.

Bug Fixes

• Links on the startup page don't work when DPI is set to more than 100%.

New Features 

• Date range data provider.
• New Change Tracking option Track Deletes. This option is turned on by default.

Changes

• Input data from non-database sources is now loaded automatically when editing a command that depends on the input data

Bug Fixes

• Combining the options Change tracking and Old data option Keep All and Export should not be allowed.

Bug Fixes

• Internal frames are sometimes incorrectly detected as external frames.

New Features 

• Content Grabber now detects if it should load and parse external frames.

Bug Fixes

• Unable to set form fields with text containing some special characters.
• Number Range data provider does not work when debugging.

Bug Fixes

• Content Grabber may crash when processing certain AJAX requests.

Bug Fixes

• Post and header URL parameters don't work when using a web browser.

New Features 

• The Request Editor now has the ability to test requests.

Bug Fixes

• Content Grabber may crash when debugging Exit or Retry commands.

Changes

• Minor user interface corrections.

New Features 

• Easier trial activations.
• Support for multiple hosts per serial key.

Bug Fixes

• User Interface displays incorrectly on some screen resolutions.

New Features 

• New Web Request editor makes it easier to create complete web requests that include URL, post data and headers.
• When clicking on AJAX URLs in the Browser Activity screen, the Web Request editor will open and provide info about post data and headers.
• New Filter selection tool.
• Exit Or Retry commands now have an option to delete the current data row when exiting.

Bug Fixes

• Creating list selections while inside a Web Element List command sometimes does not work correctly.

New Features 

• New option to capture link text when creating a new Link command. 

Bug Fixes

• Content Grabber may crash when a page is not loaded correctly.
• Content Grabber may crash when exporting data to CSV if the agent contains a capture command with content type DateTime and the captured value is an invalid date.

Bug Fixes

• Minor layout issues on script dialog windows.
• Data replacement does not always work for Regex scripts.

Bug Fixes

• Content Grabber may crash when executing a command with a selection condition if the selection does not exist.
• Some icons are missing from the command tree when debugging an agent.

New Features 

• New application menu View HTML displays the raw HTML of the currently selected browser tab.
• The JSON Transformation windows now allows testing transformations.

Bug Fixes

• Minor layout fixes in the Content Grabber editor.

Bug Fixes

• Content Grabber crashes when using the option Close Browser After Use on the Agent command.

Bug Fixes

• Trial activation screens are displayed incorrectly on some screen resolutions.
• The free Nohodo proxy plan is not automatically configured in Content Grabber for trial users.

Bug Fixes

• Page Transformation commands cause agents to fail.

Bug Fixes

• Form Field commands not executing an action can cause problems for some sub-commands, such as Download Document commands.
• Page Transformation commands often does not work correctly.

Bug Fixes

• Condition script documentation is missing from the manual.

New Features

• New flow control commands If..ElseIf..Else and Exit or Retry.
• New command Remove Duplicate.
• New command Resolve CAPTCHA.
• New command Manual Navigation.

Bug Fixes

• Visual Studio templates don't work after switching from 32bit to 64bit.

Bug Fixes

• An agent may stop or hang when retrying a URL using a Dynamic browser.

API Changes

• The ListNodes property has been removed from the ScriptParameters class.

Bug Fixes

• The options Separate output and Export Method are not synchronized in the configuration panel for Group commands, Group in Page Area commands and Web Element List commands.
• Nested Web Element List commands sometimes don't work properly.

Bug Fixes

• Data export to a separate table sometimes produces incorrect data.

New Features 

• Agent schedules can now be exported and imported from the Manage Schedules screen.

Bug Fixes

• The Save As file menu is missing.
• Content Grabber crashes when trying to delete the internal database while no internal database connection is selected.

Bug Fixes

• Unable to select web elements in the web browser on some computers.
• Unable to start the Content Grabber Windows service.
• Self-contained agents don't work correctly.

Bug Fixes

• Unable to set the option SQLite Sync.

Bug Fixes

• Unable to cancel pagination set selection when creating a Pagination command with a pagination set selection.

New Features 

• When opening a trial agent in a full version of Content Grabber, a message is now displayed to ask if the agent should be converted to a full agent.

Bug Fixes

• The installer always displays Professional edition, even when installing a Premium or Server edition.
• When creating a 32-bit runtime package on 64-bit Windows, a message will say that a 64-bit runtime package was created, even though a 32-bit package was in fact created.
• Unable to add agent commands in the 2.7.0 trial version.

IMPORANT: Older versions of Content Grabber cannot be upgraded to 2.7.0. All previous installations of Content Grabber must be uninstalled and the full version of Content Grabber 2.7.0 must then be downloaded and installed.

New Features 

• The software installers for the Premium, Professional and Server editions are now consolidated into a single installer.
• Content Grabber now installs as a 64-bit application on 64-bit Windows.
• The Content Grabber runtime is now available in both 64-bit and 32-bit versions.

Bug Fixes

• Unable to set the option SQLite Sync when using SQLite as internal database.
• When using SQLite as internal database, the option SQLite Sync is set to true by default instead of false as intended. This can result in some agents running much slower than they otherwise would.