The AppScan scan file size depends on the site content and structure (such as, number of parameters and their various values, responses size, number of cookies and their values, and so forth).

AppScan holds most of the information on disk in the Microsoft® Windows® TEMP directory and only part of the scan in memory. Every component of the scan has an impact on the size of session while scanning and the final session file.

This can affect the amount of data placed in the TEMP folder:

• Explore responses: AppScan saves the full explore response
  
  
• Test requests (potential vulnerabilities): not just the number of them but also the size of the requests (large POST requests will consume more space)
  
  
• Test responses: AppScan saves the full test response for each test even if it is duplicate, as well as all the connections between the AppScan entities and responses
  

• AppScan entities: hosts, directories, parameters, cookies
  
  
• Strings: parameter names for example
  
  
• A complex site with many hosts and complex directory structure will result in high memory consumption

1. If Rational AppScan Standard does not track the Session ID correctly, it will frequently fall out of session. In order to be able to track this particular SessionID, the following steps need to be configured under Scan Configuration > Parameters and Cookies > Advanced: Custom Parameters:
   1. Add a new Custom Parameter (use the plus sign, + , in the top right)
   2. In the Add Custom Parameter dialog, enter a name for the Custom Parameter rule in the Reference Name field
   3. Enter the regular expression needed in the Pattern field (Example: (abc[a-zA-Z0-9]+) )
   4. Leave Value Group Index and Name Group Index settings as default
   5. Select Path as the Location value
   6. Click OK to save and apply the changes
      
      
2. This creates the rule by which Rational AppScan Standard can recognize the Custom Parameter. Once this has been completed, it is now necessary to enter the parameter and set its value to be tracked under the Parameters and Cookies tab of the same Scan Configuration entry by doing the following:
   1. Add a new Parameter (using the plus sign, + , in the top right)
   2. Set the Type to Custom Parameter
   3. Select the Reference Name from the drop-down list for the previously defined rule
   4. Enable the Track this parameter during scan check-box
   5. Set Track Type to 'Login Value (Recommended)' or Dynamic as appropriate
   6. Click OK to save and apply the changes
      
      
3. With both of these steps complete, a re-scan of the application is required from Scan > Re-Scan > (Full Scan or Re-Explore).
   
   
4. Finally, if the login sequence includes a URL that contains the in-path SessionID, then it is necessary to set-up the Custom Parameter rule first before recording the login sequence. This may require that any existing login sequences be re-recorded so that Rational AppScan Standard can track the SessionID properly.

When recording a multi-step sequence that requires a form. parameter, the parameter value cannot be changed. This means that once a value has been entered in a particular form. field, the value cannot be altered. This causes a problem during the testing phase, due to the same form. being submitted multiple times, leading to an error because the value is not unique.

Example:

On page A, there is a form. field that requires a unique value. If page A is visited again, the same value cannot be entered. If the value is not unique, then you do not get to page B.

It will be helpful to get AppScan Standard Edition to enter a unique value every time it visits this page.

1. Record the multi-step sequence in AppScan
   
   
2. Export the result and save it locally
   
   
3. Open up the saved file in a text editor

Modifying the Values

Setting a parameter as Sequence Variable is done by modifying the values of the problematic parameters in the sequence text file described above.

Example:

If the user wants to modify all pages in their sequence where "Nick0001" was entered as a value, they can then define a wildcard to say enter "Nick" followed by a random 4 digit integer as a value for any parameters that uses it in the sequence. The user will find and replace all relevant instances of the "Nick0001" with their "Sequence Variable" entry in the sequence file.

The "Sequence Variable" will be a global setting so that when an instance of a particular explore sequence is running, the new value will be used throughout the sequence. The next time a new instance of the same sequence is running (for the next test), a new value will be generated, and so on.


Parameter Definition Example

Here is an example of parameter definition (from the sequence file) that contains "Sequence Variable":



In this example, the name of the sequence variable is itemID and the value that will be sent is a 5 characters random string.

To append a random string to the end of a parameter the definition would look like:



When AppScan submits the _%24RokId parameter, its value looks like

1. __SeqVariable__[variable id]__random_integer(min,max)__
   
   This will permit the user to request a random integer to be used, as well as the minimum and maximum acceptable values.
   
   
2. __SeqVariable__[variable id]__incrementing_integer(min,increment)__
   
   This will permit the user to request an incrementing integer to be used, as well as its starting value and the amount to increment (will be incremented from the value used in the previous instance of the explore).
   
   
3. __SeqVariable__[variable id]__random_string(length)__
   
   This will permit the user to request a random string to be used, as well as its length.
   
   
4. __SeqVariable__[variable id]__date_time()__
   
   This will permit the user to request a date/time stamp to be used. It will be in the following format and consist strictly of integers: MMddyyHHmmss
   
   Where:
   MM is the 2-digit current month of the year (i.e. 04 for April)
   dd is the 2-digit current day of the month.
   yy is the 2-digit current year (i.e. 07 for 2007)
   HH is the 2-digit current hour of the day in 24 hour format (i.e. 17 for 5 pm.)
   mm is the 2-digit current minute of the given hour.
   ss is the 2-digit current second of the given minute.
   
   Example:
   
   If the current time is 30 seconds past 5:52 pm on April 9th, 2007, the string will be as follows:
   
   040907175230
   
   
5. __SeqVariable__[variable id]__ date_time_milliseconds()__
   
   This will permit the user to request a date/time stamp with milliseconds to be used. It will be in the following format and consist strictly of integers: MMddyyHHmmssSSS
   
   Where:
   MMddyyHHmmss is defined the same as the date/time stamp in iv) above.
   SSS is the 3-digit current millisecond of the given second.
   
   

Here are the most common causes for varying results between scans:

Cause 1

By default, AppScan runs its automatic explore in 10 threads. Sometimes, due to timing, the threads may explore the site in a slightly different fashion from one scan to the next.

Cause 2:

Communication problems can lead to scans tests being skipped.

Cause 3:

Sometimes an application may return a different response from an identical request.

For example, AppScan may be testing a web application that is accessed through a proxy which for whatever reason periodically returns errors. This then causes inconsistent vulnerability results.

The easiest way to determine if tests are being skipped, is by opening AppScan's new real-time scan log (Scan > Scan Log) during the course of the scan. If tests were skipped, you will see entries similar to the following examples:

• Cannot connect to host: [IP/HOSTNAME]
  
  
• Test [TEST_NUMBER] [TEST_NAME] failed due to communication error: [PATH]
  
  
• Connection established with host: [IP/HOSTNAME]

Diagnosing the problem for Cause 3:

The way to isolate the issue is to run two identical scans with the traffic log enabled (see Technote 1287440 for instructions), save the Traffic log after each scan, then determine the scan differences in the Issues view.

Solution for Cause 1:

If the visited URLs displayed under Application Data > Visited URLs differ in two comparable scans, you can set up AppScan to run in 1 thread (Scan > Scan Configuration > Connection > Number of Threads). This will ensure that AppScan will traverse the site in the same fashion as a previous scan, permitting that the site content hasn't changed in the interim.

Solution for Cause 2:

If you notice excessive entries similar to the above, refer to technote 1283298 - How to address communication problems during a scan in AppScan

Solutions for Cause 3

1. Once one issue shows up in one scan, but not in the next, copy the exact test payload in the scan that came up positive. (For example, if the payload is delivered on a parameter and the Test Request is a GET, copy the first line of the Test Request to your clipboard; otherwise, if its a POST request, copy the parameters which include the payload.) Then search the Traffic log for the scan which the vulnerability was missed in and when you find the request, find the response which is the next one for the listed thread id (like the thread id for "---- Begin Thread 2768[132c] ----" is 132c) to determine why it is different from that listed in the vulnerable scan.
   
   
2. If using AppScan 7.5 or higher, an alternate solution is to perform. a similar analysis by enabling the invulnerable Variants (under the Scan menu) and searching the results for responses which do not come up as vulnerable.

• CPU - 1GHz and above
• RAM - 2GB and above
• Free Disk Space- 3-4GB
  
  Note: Ensure that the temp directory as defined in the environment variables is pointing to this drive.
  

• Do not change the default explore settings (Redundant Path and Depth Limits), most of the site will be covered with default setting and increasing the limits might introduce redundancy and increase the size of the session.
• Save the session after the Explore phase (One can even create a Business Process of the Explore phase).
• Check the following parameters:
  1. Number of visited links
  2. Number of potential vulnerabilities
  3. AppScan memory consumption (In task manager)
  4. AppScan files in the Temp directory (Extensions - DBF, CDX, FPT)If any of these parameters are extremely high or the combination of them looks out of the ordinary, you should consider running the test phase by breaking them up into groups of smaller ones (see "How to Split the Test Phase" below).

• Large number of visited links - greater than 10,000
• Normal number of visited links but a large number of potential vulnerabilities
• More than 100,000 potential vulnerabilities but no more then 500-1,000 visited links; this can be an indication of a large number of form. fields in the session.
• Memory consumption of more than 500MB
• Temp files larger than 1.5GB

• Change the name of the scan and save it again. This will create a copy so the baseline session will not be affected.
• Go to Scan > Scan Configuration and click on Test Policy
• For each scan file created (above), edit the test policy as applicable
• Go to Scan > Re-Scan > Re-Test - there is no need to explore the site again
• Run the test - you will notice that only the specified test groups are executed
• Save the session
  

• Changing the Numbers of Threads (Scan Configuration > Communication and Proxy)
• Changing the Timeout (Scan Configuration > Communication and Proxy)
• Changing the Login Sequence (Scan Configuration > Login Management)
• Adding a Custom Error Page (Scan Configuration > Error Pages)
• Adding Exclusions (Scan Configuration > Exclude Paths and Files)
  
  

To re-run a scan or scan stage:

• On the Scan menu, click Re-Scan and then click an option from the sub-menu: Full Scan, Explore, or Test.

1. Scan Configuration > Communication and Proxy
   
   Check to see if the proxy settings need to be entered, or if Rational AppScan Standard is using the Use Internet Explorer proxy settings option. If this is the case, try entering in the proxy information into the Use custom proxy settings option.
   
   Note: Your proxy settings can be found by going to Control Panel > Internet Options > Connections tab and clicking on the LAN Settings... button or contacting your Networking Administrator
   
2. Scan Configuration > URL and Servers
   
   Double check your URL structure to see if the Treat all paths as case-sensitive option needs to be turned on. If this option is de-selected and the paths are case-sensitive, Rational AppScan Standard might not be able to access the page properly through Manual Explore.
   

The Rational AppScan Standard product provides users with several indications of the areas and parts of the web application that were covered during a scan.

Application Tree

The Application Tree (in the Rational AppScan Standard user interface), is a graphical representation of the areas that were discovered and explored by Rational AppScan Standard. Users can validate that the whole application was covered by viewing this tree and making sure that no application segments were left undiscovered.

Application Data

The Application Data view (View > Application Data), is a repository of information and data about the structure and contents discovered about the application during the explore phase. This data contains:

• Visited URLs: URLs that were visited during the explore phase
• Script. Parameters: Input parameters sent to the application, such as text fields, radio button values, hidden parameters, link parameters...
• Interactive URLs: Forms which were not automatically submitted and require user interaction to be fully explored
• Broken Links: Links that Rational AppScan Standard cannot retrieve (either because they are missing, or because the application returned an error during the explore phase)
• Filtered URLs: URLs that were not explored due to explore filters (Example: path exclusions, file type exclusions, depth limit...)
• Comments: HTML comments extracted from each page that were discovered during the explore phase
• JavaScripts: JavaScript. code that was extracted from each page that was discovered during the explore phase
• Cookies: HTTP cookies that were used during the explore phase (set either by a "Set-Cookie" header, or by client-side technologies such as JavaScript)