Microsoft Knowledge Base Article
This article contents is Microsoft Copyrighted material.
©2005-©2007 Microsoft Corporation. All rights reserved.
Terms
of Use |
Trademarks
Article ID: 291844 - Last Review: October 16, 2002 - Revision: 1.1
PRB: ISAXXMLReader::parseURL Consumes Large Amount of Memory Over HTTP
This article was previously published under Q291844
When you use code such as the following, note that the
parseURL method of the
SAXXMLReader object consumes an unexpectedly large amount of memory when parsing a large XML file over HTTP:
Set saxReader = CreateObject("Msxml2.SAXXMLReader.3.0")
saxReader.putFeature "server-http-request", true
saxReader.parseURL "http://localhost/big.xml"
You expect that the
parseURL method would not consume much more memory than the actual size of the XML file.
The current implementation of the WinHttpRequest component that is used by the
ServerXMLHTTP object does not support "pull" semantics for retrieving the response data; that is, the WinHTTPRequest does not support fetching the response off the wire in chunks. As a result, the response data gets double buffered in the WinHttpRequest component.
The component itself maintains one copy internally and then makes a second copy to make the
responseText,
responseBody, and
responseStream properties available for use with scripting languages. The double buffering may lead to large memory footprints when using the Simple API for XML (SAX) or Document Object Model (DOM) components to process large XML files over HTTP.
Microsoft has confirmed that this is a problem in the Microsoft products that are listed at the beginning of this article.
Memory usage is reduced when you use Microsoft XML version 3.0, Service Pack 1 or MSXML 4.0.
Steps to Reproduce Behavior
- Create the large XML document about 100 MB in size by following these steps:
- In Microsoft Visual Basic, create a new Standard EXE project. Form1 is created by default.
- Add a Command button to Form1. Command1 is created by default.
- Paste the following code into the Click event for Command1:
On Error GoTo handler
Open "c:\inetpub\wwwroot\big.xml" For Output As #1
Print #1, "<?xml version=""1.0"" ?>"
Print #1, "<XML>"
Dim i As Long
For i = 1 To 100000
Print #1, "<t>" & String(1000, "o") & "</t>"
Next i
Print #1, "</XML>"
Close #1
Exit Sub
handler:
MsgBox Err.Number & ":" & Err.Description, vbExclamation, Err.Source
- Create the test VBScript file:
Paste the following code into the new text file:
To run the code with MSXML 4.0 parser, change:
Set saxReader = CreateObject("Msxml2.SAXXMLReader.3.0")
To
Set saxReader = CreateObject("Msxml2.SAXXMLReader.4.0")
- Press the F5 key to run the project from Visual Basic.
NOTE:The program creates a file of about 100 MB in size. The program takes quite a while to run. Be sure that you have sufficient free hard disk drive space. Be sure to delete the XML file after you complete the test below.
- Create the test VBScript file:
- Using Windows Notepad or another text editor, create a new text file.
- Paste the following code into the new text file:
dim saxReader
Set saxReader = CreateObject("Msxml2.SAXXMLReader.3.0")
saxReader.putFeature "server-http-request", true
saxReader.parseURL "http://localhost/big.xml"
WScript.Echo "Done! saxReader.parseURL"
- Save the text file as C:\Big.VBS.
- Open Windows NT Performance Monitor to monitor memory usage:
- In Microsoft Windows NT or Microsoft Windows 2000, press the CTRL+ALT+DEL keypad combination.
- On the dialog box that is displayed, select Task Manager.
- In Task Manager, click the Performance tab, and note the graph of Memory Usage History.
- Size and position Performance Monitor so that you can see the Memory Usage History but still work on the Windows Desktop.
- Test:
- On the Windows Start menu, select Run.
- In the Run dialog box, type the following command:
- Click OK, and note memory usage in Performance Monitor.
APPLIES TO
- Microsoft XML Parser 3.0
- Microsoft XML Parser 3.0 Service Pack 1
- Microsoft XML Core Services 4.0
Community Feedback System
Very often, it takes hours to solve a problem. Very often, you've looked high
and low, and have tried a lot of solutions. When you finally found it, chances
are, it was because someone else helped you. Here's your chance to give back.
Use our community feedback tool to let others know what worked for you and what
didn't.
Please also understand that the community feedback system is not warranted to be
correct, it's simply a system that we've built to let people try and help each
other. If something in a feedback response doesn't make sense to you, or you're
not comfortable making changes that the feedback talks about (like registry
edits), please consult a professional.
Thank you for using kbAlertz.com Feedback System.
-- Scott Cate