- Content Grabber fails to parse pages with IFrames.
- Cache Method added to agent properties. This option can be used to ensure parsers never retrieve pages from cache.
- JSON properties with null values are ignored by the JSON Parser.
- Screenshot log message is displayed every time debugging completes.
- Added button to open exported Excel spreadsheet to the Run screen.
- Added New Line Encoding to CSV export.
- File name transformation does not work for Download Page commands.
- Changing a Condition Script doesn't trigger an indication that the agent has changed.
- The form field input value is not passed to exec events when executing a Form Field command.
- CSV file export now uses the .txt file extension by default when values are delimited by tab.
- The file encoding for CSV export causes problems when BOM is not included in UTF-8.
- Content Grabber editor performance improvements.
- An agent can now be configured to not export data.
- The file encoding for CSV export is set to UTF-8 BOM by default which causes problems in some applications. The encoding should be changed to UTF-8 without BOM.
- Incorrect icon for Download Image commands.
- The Referer request header is set incorrectly.
- The transformed capture panel on most configuration screens cannot be resized.
- A description can now be added to commands, which will be displayed on the command info panel.
- Command transformation scripts fail when used in a custom script library.
- Unable to set a page load percentage as a success criteria.
- Agent loading screen added to self-contained agents.
- Minor performance enhancements.
- The message displayed when saving data export settings in self-contained agents has changed to make it clear the agent needs to be rerun to export data in a new format.
- The start button in self-contained agents doesn't become inactive immediately after pressing it.
- Add Sub-Command and Edit buttons added to command info panel.
- Unable to navigate files on the local file system, because file URIs are calculated incorrectly.
- Export Row Count now available in Post Status.
- Agent multitreading options have changed to \"Max Active Browsers\" and \"Max Active Parsers\", and now more accurately controls the number of active parsers and dynamic browsers.
- Email notifications include the wrong value for Export Row Count.
- The User Agent is not always set correctly for HTML Parsers at design time.
- The manual is missing a topic about the Post Status functionality.
- Custom Content Grabber Regex operations fail if they are trailed or preceded by spaces.
- The default script library C# project has the Condition Script unit test in a wrong name space.
- Post Status does not work correctly at runtime.
- Condition script is missing from the default script library.
- The HTML Parser uses a different User Agent than the dynamic browser by default.
- Unable to load a specific URL into a browser tab in some situations.
- Incorrect scaling on many screens in the Content Grabber editor when using high DPI settings.
- Post Status does not always work correctly at runtime.
- Added Post Status screen.
- HTML Parser User Agent is not set to the same user agent as the web browser by default, which causes issues on some websites when switching from dynamic browser to HTML Parser.
Added Notify on successful data extraction and Success Email Addresses to the notification screen.
- Some agents export duplicate data when they shouldn't.
- Some websites fail to make AJAX requests in the Content Grabber editor, which may cause the website to load incorrectly.
- Continue & Retry Errors does not work correctly on the screen that shows all performance sessions.
- Setting the Default values check box when editing a URL command does not disable data transformation.
- Improved reliability and performance when multi-threading.
- Page load success criteria don't work correctly.
- An error may occur when executing a Form Field command that doesn't execute an action.
- Potential thread handle leak when multi-threading. This has the potential to cause stability issues when executing many high performance agents on a single server.
The Continue & Retry All button on the screen that shows all performance sessions, will appempt to start sessions that are complete and have no errors.
- Performance session progress added to screen that shows all performance sessions.
- Added Continue & Retry All button to screen that shows all performance sessions.
- Added Start All button to screen that shows all performance sessions.
- Added Stop All button to screen that shows all performance sessions.
- Added unescape_string method to Regex transformations.
- The JSON Transformation Script Editor does not use the raw page content as input data, which can make it difficult to generate a correct transformation.
- Unable to overwrite an existing agent in some situations.
- The script property DatabaseConnection is not available in Data Export scripts.
- The method WriteDebug does not work correctly in Raw Data Export scripts.
- Changing debug speed during debugging doesn't take emmediate effect.
- The script library panel is expanded by default on the Script Editor screen.
- New property HasDoneAction available in most scripts.
- New methods in the RuntimeData Script Utilities class.
- An error occurs when trying to import an agent or upgrade Content Grabber if the computer's TEMP or TMP environment variables are configured incorrectly, or points to paths that are inaccessible.
- Unable to upgrade templates.
- Content Grabber icons are not associated with Content Grabber files.
- Command and script templates can now be referenced instead of copied into an agent.
- Added suggested command types to the New Command drop-down panel.
- Added Search button to the View Data screen.
- Templates created with versions lower than 2.33.0 are no longer available, but can be upgraded to version 2.33.0+ from the Tools menu.
- Templates are now stored in individual files instead of one big file. This makes it easier to copy or move templates between computers.
- Some menu items have been moved from Application Settings to Tools.
- Retrying commands doesn't always work as intended.
- Logging does not work in self-contained agents.
- Input data is always enclosed in double quotes in self-contained agents, even when it's not required.
- Some web forms that shoud be able to be processed with a HTML Parser cannot.
- Error downloading files from direct HTML pages.
- Content Grabber crashes when pagination limit is set to one page.
- The Form Field option Set Form Field Value has no effect. This causes checkboxes to be set incorrectly most of the time.
- The action option Scroll to end of page does not always work correctly.
- The Content Grabber editor crashes when trying to open a link in a new browser if that link does not start an action and the browser does not already exist.
- A page load error is not reported when an action command configured to open a new web browser fails to open a new web browser.
- Unable to enable Min. page loads on Agent success criteria screen.
- Unable to change default JSON export settings.
- The Wait for Content timeout value keeps increasing automatically every time an agent is loaded.
- Error running an agent using MySQL as internal database.
- A database lock error may occur when running multiple sessions.
- Switching from Dynamic browser to HTML Parser can make it impossible to configure an action command to load a page in a new browser.
- Incorrect status information while running an agent.
- Performance optimizations.
- Content Grabber is unable to handle websites that make exact copies of web elements. This is the case for certain areas of amazon.com.
- Error retrying Pagination commands.
- New agent option Delay Between Performance Sessions can be used to specify a delay between session startups when starting multiple performance sessions at once.
- The RuntimeData class is not documented in the manual.
- Performance improvements when using SQLite or SQL Server as internal database.
- Performance improvements.
- Performance improvements when adding many data rows per loaded page.
- It's no longer required to execute a Web Element List to be able to select web elements inside the web element list.
- No data is processed when a performance session specifies a data set larger than the total number of data entries.
- The action timeout value File Download Started has no effect and a default value that is often too low is used.
- An agent may exit prematurely after retrying a Link command.
- An error may occur when Continuing an agent.
- The Agent option Keep Connection Alive has not effect.
- Added Use Test Set to the Agent Explorer context menu.
- Error handling options on Download Document commands have no effect.
- Added Test Sets to data provider configuration, which makes it possible to run tests on a limited data set without having to change data source.
- Proxy API configuration doesn't always work correctly.
- Some scripts are not executed correctly at design time when using the option Always Update Design Value.
- Status does not switch to the first session when starting all performance sessions from the Run screen.
- New option Always Separate Session Export Data can be used to ensure all session data is always exported separately.
- New commandline and API parameters agent_import_path and agent_import_name can be used to control to where an agent is imported when running an exported agent.
- Performance sessions now always log to different log files.
- The Content Grabber runtime will now always log information to the agent's log folder or the specified log path. The runtime will no longer create default folders in the current user's document folder.
- The default Data Export script has an error.
- The Session ID available in data export scripts is always null when executing Data Export scripts from the Content Grabber editor.
- An error occurs when downloading a file if the web server does not provide a suggested filename.
The Session ID is now available to all scripts.
Action error handling now has an option to rotate, and optionally remove, the current proxy.
Exit or Retry commands now have an option to rotate, and optionally remove, the current proxy on retry.
Exit or Retry commands now have an option to delay retrying.
- New option Log error and exit on failure on Exit or Retry commands
- The option Delete on exit on Exit or Retry commands can now be used when both retrying and exiting.
- A column retry_count_ has been added to internal data tables. This means that some internal tables will need to be re-created, and agents that rely on internal data will be reset. To avoid the re-creation of the internal data tables, a column retry_count_ of type int can be manually added to the tables.
- Retrying list commands does not work correctly.
- An error may occur when debugging an agent with an Exit Or Retry command.
- The Retry Count property on Exit Or Retry commands are not saved properly.
- Added MPP proxy API support.
- Error testing proxy APIs.
- The Content Grabber editor hangs or crashes if an application proxy error occurs at startup.
- Added runMethod option to the simple Web requests.
- An error may occur when resuming agents using the API.
- An error may occur during file download.
- Server wide default paths can now be specified.
- When importing an agent that already exists, a new name can be specified for the imported agent.
- Multiple agents can now be exported at once.
- Content Grabber browser updated to new version of Chromium.
- The Block URLs option has been moved to agent settings, and is now applied agent wide. Also, multiple regular expressions separated by line break can be specified.
- Some websites that change page URLs without loading a new page are processed incorrectly.
- Converting a Visual Web Ripper project may crash Content Grabber.
- Logs can now be filtered by log level.
- Log level is now added to file logs.
- File logs are now tab delimited, so they can be loaded into Excel or other programs that can handle tab delimited files.
- All agent sessions or a specified range of agent sessions can now be started, viewed or deleted from the Run screen.
- Added support for Proxy APIs.
- The 3rd party proxy provider Proxicity has been removed. Proxy API with Proxicity as provider can be used instead.
- Insufficient log information when an agent fails to start.
- Web page redirect sometimes causes an error when using the HTML Parser.
- All possible agent exit codes are not listed in the manual.
- New method SetPageLoadError is available to most scripts.
- New Selection Missing option Log Page Load Error and Ignore Command.
- Command names containing single quotes sometimes causes an error.
- The Page Load Completed timeout has no effect on page loads in a HTML Parser at design time.
- Page load errors in the HTML Parser are sometimes lacking details about the error.
- The action option Block URLs has no effect.
- The export option Export Empty Row If No Data is now set to true by default.
- Changing options on a Wait for Content command does not always activate the Save quick bar menu.
- Some large web pages are not loaded correctly in a HTML Parser at design time.
- An Execute Script command can now remove the current proxy if the agent uses a proxy list.
- Commands can now be copied/pasted between agents in different Content Grabber editors.
- Multiple commands can now be added at once by specifying multiple command names separated by comma. Command names separated by new lines or tabs will be automatically converted to comma separated names when pasted into a command name field.
- Pagination commands fails if a command inside the pagination loop refreshes the page containing the pagination link(s).
- Content Grabber fails to download an image if a target web server suggests an invalid file.
- An incorrect warning is displayed every time a command is moved within the same browser sub-tab.
- Agents using performance sessions don't export any data.
- Content Grabber is unable to detect some methods in a default script library.
- Unable to scroll a web page in the Content Grabber editor by dragging the scroll bars.
- Web pages that change the page URL without loading a new page are not always processed correctly.
- Manually removing internal data now also removes all agent status information.
- Performance sessions with IDs containing odd numbers, or processing a data list with an odd number of entries, fail to loop through the entire data list.
- An agent exporting data to files and using performance sessions will now always export data from all sessions into a single file. An agent session will only export data if no other sessions of the same agent are currently running.
- The info panel on the Command Properties configuration window is now sized more appropriately.
- Extracted data is sometimes exported incorrectly to Excel and CSV when converting rows to columns
- The Content Grabber editor sometimes fail to track DOM events.
- Inline DOM events on delay loaded content are never tracked.
- The action event click() does not fire mouseenter and mousemove DOM events.
Navigate in Browser toolbar button does not work correctly in the Content Grabber editor.
- Content Grabber may crash when canceling a file download in the editor.
- Old browser processes are not closed properly when opening a new agent.
- Restoring an agent backup does not work properly.
- Resolve CAPTCHA templates generate incorrect commands.
- Most scripts now have access to a new property RuntimeData which provides access to internal data while an agent is running.
- Simple web requests to the Content Grabber service don't work.
- Extracted boolean values are sometimes handled differently when designing and running an agent.
- Custom assemblies are not loaded correctly when used in scripts.
- Some pages are loaded incorrectly when using a JSON, HTML or XML parser.
- Some JSON content is not parsed correctly by the JSON parser.
- The Content Grabber web browser does not work on some Windows configurations.
- Added default logging options for agents.
- A script library can now be any .NET assembly.
- A custom parameter can not be passed to methods in a script library.
- New script parameter IsParentCommandMissingSelectionOrData. This parameter is set to true if the parent container's selection was not found, or is iterating through an empty data list.
- When an agent is run from the commandline or API, the agent now logs to files by default.
- Content Grabber is unable to process local files.
- The export option Export empty row if no data does not work correctly.
- The Content Grabber web browser fails to load pages on some computers.
- Added support for browser zooming in the Content Grabber editor.
- Help file updates.
- Content Grabber sub-processes may not shut down properly if Content Grabber crashes or if it's process is terminated.
- An error may occur when loading a URL with the HTML Parser.
- Added option to delete website cookies in the editor.
- Cookies are not always set correctly in the HTML Parser.
- The ImportSilent method may import an agent incorrectly.
- Remove Duplicate commands don't work properly on DateTime data types.
- The ImportSilent method may import an agent incorrectly.
- Content Grabber can now start multiple agent instances at once if the agent supports performance sessions.
- An agent will try to process multiple list commands in sessions if more than on command has the option "Process in Session" set to true.
- Content Grabber sometimes mark visible web elements as if they were hidden.
- Added Performance Sessions which makes it easier to get multiple instances of the same agent to work on the same task.
- Added action event key that takes a character as parameter.
- Old action event key has been renamed keycode.
- Unable to to clear the Pagination command option Limit page count with fixed value.
- Added agent option to not load style sheets. When using a HTML Parser all inline styles will be removed as well.
- Some valid JSON content cannot be loaded in the JSON Parser.
- New override of API method ImportSilent makes it possible to import an agent to a specified location with a specified name.
- New API method DeleteAgent.
- New CSV export option Export Keys.
- Some buggy websites, that load more content on scroll, make it possible to indefinitely scroll when a browser window is at a certain height. This can cause an agent to run indefinitely.
- Added Google Drive distribution option.
- Added DropBox distribution option.
- Added JSON export target.
- Added options Has Headers and Ignore Parse Errors to CSV data providers.
- Incorrect default document converter script.
- Selections are not always marked correctly in the editor browser.
- Unable to create new shared script libraries from an agent that is not located in the default agents folder.
- Downloaded files are not distributed when exporting to Excel 2007+.
- Empty columns are inserted when embedding images in Excel 2007+.
- New API setting AgentStartTimeout can be used to specify the number of seconds an agent is allowed to start processing before a timeout occurs.
- Additional logging when an agent fails to start.
- New setting RuntimePath can be used in the API to specify the location of the Content Grabber runtime. If this setting is not specified, the runtime files must be copied to the same folder as the host application executable.
- The Create Website Images button has been moved to the Tools menu.
- New feature Create Website Images can be used to take screenshots of all types of web pages an agent visits.
- A warning message is now displayed when viewing data from an agent that converts rows into columns, since this functionality is only applied when exporting to file formats, and is not applied when viewing extracted data.
- The Page to PDF command type has been renamed to Download Page and can now download a page as PDF or HTML.
- Remove Duplicate commands don't work when debugging.
- Screenshot commands don't work properly on right-to-left language websites.
- Screenshot commands don't always work properly when Size of text, apps and other items is set to more than 100% in Windows.
- Agents using the obsolete database export option Add To Existing Data may cause an error when exporting.
- Agents with command names containing some special characters may cause an error.
- Documentation for Wait for Content commands is missing in the manual.
- The Content Grabber editor may crash on some script syntax errors.
- New command type Wait for Content can be used to force the parent action command to wait for specific URLs or web content.
- The Action properties Wait For Regex and Wait For Urls have been combined into a single property Wait For Content.
- The Action properties Wait For Regex Timeout and Wait For Urls Timeout have been combined into a single property Wait For Content Timeout.
- Base meta tags are not always processed correctly when using a HTML Parser.
- Session cookies are no longer removed on agent start unless specifically configured in the agent command.
- Content Grabber crashes when trying to import a Visual Web Ripper project created with a recent version of Visual Web Ripper.
- The Delete button in the File menu does not work.
- Values from Local Storage and Session Storage are now available using the Page Attribute command.
- Content Grabber waits too long on some websites causing agents to run slower than they should.
- Custom HTML tags are not handles correctly.
- The option to limit pagination page loads with a fixed value is not saved correctly.
- New option to limit pagination page loads with a fixed value.
- Crash when using a HTML Parser and setting a form field in design or debug mode.
- Crash when setting an Exit or Retry command to use a specific command and then not specifying a command.
- Error exporting data to file formats when converting rows to columns and the converted list command generates no rows.
The Wait for XPath option now allows page loads while waiting for an XPath selection to exist.
- Error executing Page Transformation commands in design and debug mode when loading content directly into HTML parsers.
- Error exporting historical data to MySQL.
- HTML Parser requests never times out, no matter the configured time out value. This is a serious issue, especially when processing a large number of different websites, because some websites never return a response, and Content Grabber will then hang until stopped manually.
- Number of threads used when running an agent is not always correct when using HTML Parsers.
- It's not possible to see if a script is turned on or off on all command configurations screens.
- An error may occur at design time when converting a document that opens in a new window.
- "base" HTML tags without href attributes cause HTML parsers to generate wrong URLs.
- Database connection details saved in an agent maybe duplicated.
- An error may occur when a web page refreshes in a dynamic browser.
- New option to automatically add a capture command when creating a new Form Field command.
- Agent may fail to export when change tracking is turned on.
- CSV file content can now be edited directly from the Data Provider configuration window.
- Unable to cancel anchor selection mode.
- Exporting data since last successful run when using change trackikng does not work correctly.
- An error occurs when choosing captured data as data provider.
- New action option Ignore error codes can be set to false to force an error when a website returns an error code, even when the website returns content. Content Grabber ignores status codes by default when content is returned, because some websites return incorrect error codes, or return error codes that have special meaning and should be handled by the Content Grabber agent.
- HTML parsers no longer fail by default when an error code is returned from the web server, unless the option Ignore error codes is set to false. This makes the behavior the same as when using a dynamic browser. WARNING: this may cause existing agents to behave differently unless you set the option "Ignore error codes" to false.
- The ScriptUtils function GetData() makes a POST request instead of a GET request.
- New function insert_data is now available for Content Transformation regex scripts.
- When certain errors occur in Content Grabber and an agent process is restarted, web browser processes may be left running and never shut down.
- Page to PDF commands may fail depending on timing.
- New option Deliver data files can be turned off when distributing data, so only extracted files are distributed.
- New option Always update design value can be used on capture commands to make sure their design values are automatically updated when a new web page is loaded.
- When posting data with a URL, the pattern [CG_FILE_UPLOAD_PATH:...] can be used instead of the actual file content when upload a file.
- The option Deliver Extracted Files has no effect when delivering extracted data to email or FTP.
- Missing files may cause data export to fail when delivering data to email or FTP.
- The options Default Email Settings and Default Email Addresses are not saved when saving the email delivery settings.
- An agent may hang on websites that continuously make AJAX calls.
- Request method can now be specified when loading a web request.
- Post, header and method parameters can now be specified in any order when loading a web request.
- Additional information added to the web request editor to make it easier to create file upload requests.
- Log settings can now be set in an Initialization script, but overriding the commandline parameters in the GlobalData object.
- Response headers are not available in scripting.
- Custom headers can now be set for all requests in web browsers.
- The property LastBrowserMessage has been replaced with LastErrorResponse in scripting.
- Screenshot commands don't work.
- Error setting proxy to Fiddler.
- The base meta tag is not taken into consideration when extracting URLs.
- Proxicity proxy support.
- Luminati proxy configuration is disabled.
- Default email settings shared by all agents on a computer.
- Messages can be sent from an agent to its hosting application when using the API.
- Outdated help file topics.
- Support for historical data when using change tracking.
- New capture command option Exclude from change tracking.
- New command type Page to PDF saves a web page as a PDF file.
- Refresh Document commands cause the debugger to hang.
- Content Grabber may crash when editing commands that consume data.
- The Max. repeats option for link commands does not work.
- The Close Browser command doesn't properly close a browser.
- Links on the startup page don't work when DPI is set to more than 100%.
- Date range data provider.
- New Change Tracking option Track Deletes. This option is turned on by default.
- Input data from non-database sources is now loaded automatically when editing a command that depends on the input data
- Combining the options Change tracking and Old data option Keep All and Export should not be allowed.
- Internal frames are sometimes incorrectly detected as external frames.
- Content Grabber now detects if it should load and parse external frames.
- Unable to set form fields with text containing some special characters.
- Number Range data provider does not work when debugging.
- Content Grabber may crash when processing certain AJAX requests.
- Post and header URL parameters don't work when using a web browser.
- The Request Editor now has the ability to test requests.
- Content Grabber may crash when debugging Exit or Retry commands.
Minor user interface corrections.
- Easier trial activations.
- Support for multiple hosts per serial key.
- User Interface displays incorrectly on some screen resolutions.
- New Web Request editor makes it easier to create complete web requests that include URL, post data and headers.
- When clicking on AJAX URLs in the Browser Activity screen, the Web Request editor will open and provide info about post data and headers.
- New Filter selection tool.
- Exit Or Retry commands now have an option to delete the current data row when exiting.
- Creating list selections while inside a Web Element List command sometimes does not work correctly.
- New option to capture link text when creating a new Link command.
- Content Grabber may crash when a page is not loaded correctly.
- Content Grabber may crash when exporting data to CSV if the agent contains a capture command with content type DateTime and the captured value is an invalid date.
- Minor layout issues on script dialog windows.
- Data replacement does not always work for Regex scripts.
- Content Grabber may crash when executing a command with a selection condition if the selection does not exist.
- Some icons are missing from the command tree when debugging an agent.
- New application menu View HTML displays the raw HTML of the currently selected browser tab.
- The JSON Transformation windows now allows testing transformations.
- Minor layout fixes in the Content Grabber editor.
- Content Grabber crashes when using the option Close Browser After Use on the Agent command.
- Trial activation screens are displayed incorrectly on some screen resolutions.
- The free Nohodo proxy plan is not automatically configured in Content Grabber for trial users.
- Page Transformation commands cause agents to fail.
- Form Field commands not executing an action can cause problems for some sub-commands, such as Download Document commands.
- Page Transformation commands often does not work correctly.
- Condition script documentation is missing from the manual.
- New flow control commands If..ElseIf..Else and Exit or Retry.
- New command Remove Duplicate.
- New command Resolve CAPTCHA.
- New command Manual Navigation.
- Visual Studio templates don't work after switching from 32bit to 64bit.
- An agent may stop or hang when retrying a URL using a Dynamic browser.
- The ListNodes property has been removed from the ScriptParameters class.
- The options Separate output and Export Method are not synchronized in the configuration panel for Group commands, Group in Page Area commands and Web Element List commands.
- Nested Web Element List commands sometimes don't work properly.
- Data export to a separate table sometimes produces incorrect data.
- Agent schedules can now be exported and imported from the Manage Schedules screen.
- The Save As file menu is missing.
- Content Grabber crashes when trying to delete the internal database while no internal database connection is selected.
- Unable to select web elements in the web browser on some computers.
- Unable to start the Content Grabber Windows service.
- Self-contained agents don't work correctly.
- Unable to set the option SQLite Sync.
- Unable to cancel pagination set selection when creating a Pagination command with a pagination set selection.
- When opening a trial agent in a full version of Content Grabber, a message is now displayed to ask if the agent should be converted to a full agent.
- The installer always displays Professional edition, even when installing a Premium or Server edition.
- When creating a 32-bit runtime package on 64-bit Windows, a message will say that a 64-bit runtime package was created, even though a 32-bit package was in fact created.
- Unable to add agent commands in the 2.7.0 trial version.
IMPORANT: Older versions of Content Grabber cannot be upgraded to 2.7.0. All previous installations of Content Grabber must be uninstalled and the full version of Content Grabber 2.7.0 must then be downloaded and installed.
- The software installers for the Premium, Professional and Server editions are now consolidated into a single installer.
- Content Grabber now installs as a 64-bit application on 64-bit Windows.
- The Content Grabber runtime is now available in both 64-bit and 32-bit versions.
- Unable to set the option SQLite Sync when using SQLite as internal database.
- When using SQLite as internal database, the option SQLite Sync is set to true by default instead of false as intended. This can result in some agents running much slower than they otherwise would.