|
<< <%SKIN-STRTRANS-SYNTOC%> >> Navigation: »No topics above this level« Improving Agent Performance and Reliability |
It is important to understand that the complexity of web scraping increases significantly when extracting large amounts of data. The time it takes to perform certain operations may take so long that the whole web scraping process becomes impractical. Web scraping is unreliable in nature, and the longer a web scraping project runs, the higher the chance something fails and interrupts the web scraping process.
The following topics list a few steps you can take to increase performance and reliability. Performance tuning is not for the novice user, and if you have no IT experience, you may not be able to implement the recommendations in this topic.
•Using MS SQL as Internal Database
•Using the Static Parser Browser