You can use error notifications to warn when an agent encounters an error, so that the administrator can take appropriate action. The administrator can use the logs to get more information about the errors.
Error Notifications
Notifications are email messages that are sent to a specific email address when certain conditions occur. The notification configuration screen is available by clicking the Agent Settings > Notifications menu.
This agent sends email notifications if it encounters any
critical errors
An email notification can be sent when an agent finishes, but never during agent execution or when debugging an agent. You can choose to send email notifications when one or more of these conditions are met:
Condition |
Description |
Always send notifications when an agent completes a run |
A notification email will always be sent when the agent finishes, whether an error occurs or not. |
Send notification on critical errors |
An email notification will be sent when critical errors occur. A critical error will appear as red color on the log screen. These critical errors include page load errors and also situations where required web selections are missing from a web page. |
Send notification on unsuccessful data extraction |
A notification email will be sent when an agent run doesn't meet a defined success criteria. |
Send notification on successful data extraction |
A notification email will be sent when an agent run meets a defined success criteria. The notification is sent to Success email addresses if specified. |
A script can trigger notifications by calling one of these methods on the script parameter object:
Method |
Description |
void Notify(bool alwaysNotify = true) |
Triggers notification at the end of agent execution. If alwaysNotify is set to False, this method only triggers a notification if you configure the agent to Send notification on critical errors (see above). |
void Notify(string message, bool alwaysNotify = false) |
Triggers notification at the end of an agent execution, and adds a message to the notification email. If alwaysNotify is set to False, this method only triggers a notification if you configure the agent to Send notification on critical errors (see above). |
Success Criteria
A success criteria can be defined to tell an agent when it should consider an agent run successful. For example, a success criteria could include that the agent must load a minimum number of pages.
Define criteria for a successful agent run.
The Success Criteria window can be opened from the the Run menu in the Content Grabber editor, or from the Notification window.
The following three success criteria can be defined.
Criteria |
Description |
Min. page loads |
The minimum number of pages the agent must load. |
Min. data count |
The minimum number of data entries the agent must extract. A data entry is defined by setting the option Increase data count on any command. The data count is increased every time a command with Increase data count is executed. |
Max. page errors |
The maximum number of page errors allowed. |
Success criteria can be specified as absolute numbers or as a percentage of the value from the last successful agent run. For example, if Min. page loads is set to 95% it means the agent must load minimum 95% of the number of pages it loaded at the last successful run. A Percentage criteria always succeeds if the agent has not been successfully run before.
Success criteria can be used to decide when email notifications are sent, but are also used in Change Tracking to decide when it's safe to mark data as deleted.
Most email servers are very restrictive in regards to who or what is allowed to send emails. If you are getting an error when sending emails from Content Grabber, then it's most likely a problem with the configuration of your account. Here is a list of things you can check:
•Make sure your email account is configured to allow you to send emails from the email address specified in From email address.
•Make sure your username and password are correct, and that login to your account is not restricted. Use another email client or your web browser to login to your email account to make sure you can login.
•Use SSL if required. Most online services, such as Gmail, requires SSL.
•Use the correct port. Gmail uses port 587 for example.
•Most online services, such as Gmail, don't allow just any application to login to your email account. Many online services will have a special option to allow any application to login to your account, and you must set this option to allow Content Grabber to login. In Gmail you can set this option on the following page:
https://www.google.com/settings/security/lesssecureapps
Error Logs
Logging is set to low by default when running an agent, and high by default when debugging an agent. Debug logging is always saved to a database table, but runtime logging can be saved to a database table or a text file.
The database table supports the log viewer in the Content Grabber editor, which you can access by clicking the View Log button in either the Run ribbon menu or the Debug ribbon menu.
Content Grabber supports 3 log levels, High, Medium and Low. The High log level will log everything. The Medium log level will log errors and warnings, and the Low log level will only log errors.
The critical errors in this log are red in color
URL Column
The URL column in the Agent Debug Log window shows links to all web pages that the agent has processed, and you can double-click on any of these links to open the web page in your default browser. For any failures on data extraction, this is an easy way to open the web page and check if the web page layout is different from the expected layout.
NOTE: Double-clicking on a link to a sub-page only works if the website allows direct links into the sub-pages. Many websites don't allow direct links into sub-pages without going through other pages first. Read below to learn how to directly access the HTML for sub-pages. |
HTML Column
If a website does not allow direct links into sub-pages, you can configure an agent to save the entire HTML of all processed pages. When you configure the agent to log all HTML, the HTML column will contain buttons that you can click to view the raw HTML in the Content Grabber HTML viewer. The raw HTML does not include style sheets and other support files that are necessary to render a complete web page properly, but it will often show you enough information to determine what is causing an error (such a CAPTCHA page).