Most efficient way of writing to a *.txt file

[raheel88]

All,

What is the most efficient way of writing several lines of data to a *.txt file? Currently, I store the required text in memory then write it all to a user-specified text file at the end, similar to the following;

Dim text As String = ""
text = text + "1st line of text" & vbNewLine
text = text + "2nd line of text" & vbNewLine
text = text + "3rd line of text" & vbNewLine

Dim path As String = "C:\Temp\fileName.txt"

My.Computer.FileSystem.WriteAllText(path, text, True, System.Text.Encoding.Default)

This works well enough for up to about 1000 lines of text. However, I often need to write 3000+ lines to a single file and it is noticeably slow using the above method. Is there a more efficient way?

Thanks in advance!

[deceptikon]

File operations are inherently slow, relative to in-memory operations. I'd probably go with writing smaller blocks than dumping all of the file contents to a line for a single WriteAllText call. The framework can handle its own buffering for performance and you can limit the amount of string processing you're doing in memory as well as how much memory you're using. A big hit to performance often involves crossing cache lines, which large strings are wont to do.

The absolute fastest you can get would probably force you to drop down to WriteFile using PInvoke, but WriteAllText is one of the fastest options from the framework from benchmarks I've seen.

[tdrosiadis]

You could use the StringBuilder class and write chuncs of text to your file. As a .net class it might be less resource concuming. Havn't tryed it though. Just a though

[Reverend Jim]

Assuming that you are replacing the text in the file with the new text, I would do the following

Dim text As New System.Text.StringBuilder

text.Append("1st line of text" & vbCrLf)
text.Append("2nd line of text" & vbCrLf)
text.Append("3rd line of text" & vbCrLf)
.
.
.    
System.IO.File.WriteAllText(filename,text.ToString())

Depending on how many lines you have, repeated concatenation as in your code can get noticibly slow. StringBuilder is much more efficient. Try the following and see for yourself.

Dim text As String = ""

For i As Integer = 1 To 1000
    text &= "this is line " & i & vbCrLf
Next

and compare it to

Dim text As New System.Text.StringBuilder

For i As Integer = 1 To 1000
    text.Append("this is line " & i & vbCrLf)
Next

[raheel88]

Thanks all, what Reverend Jim suggested is the best option by the looks of it.

Having internet problems so will mark as solved when I get a chance :)

Thanks again!

[dionterry]

Best way is:

Dim sr as new system.io.streamwriter("Location of txt file")
sr.write("Text to write") 'You can also use writeline etc.
sr.close