Read a file in chunks in Ruby?

Up vote 4 down vote favorite share g+ share fb share tw.

I need to read a file in MB chunks, is there a cleaner way to do this in Ruby: FILENAME="d:\\tmp\\file. Bin" MEGABYTE = 1024*1024 size = File. Size(FILENAME) open(FILENAME, "rb") do |io| read = 0 while read Left : MEGABYTE data = io.

Read(cur) read += data. Size puts "READ #{cur} bytes" #yield data end end ruby link|improve this question asked Nov 5 '09 at 17:11user135827518314 83% accept rate.

Adapted from the Ruby Cookbook page 204: FILENAME="d:\\tmp\\file. Bin" MEGABYTE = 1024*1024 class File def each_chunk(chunk_size=MEGABYTE) yield read(chunk_size) until eof? End end open(FILENAME, "rb") do |f| f.

Each_chunk() {|chunk| puts chunk } end Disclaimer: I'm a ruby nub and haven't tested this.

Yes, this works. However, I thought that IO. Read would throw if the number of bytes left was less than chunk size.

I thought that because I had read about IO. Readbyte, which will throw TruncatedDataError. Looks like that does NOT apply to read.

An oversite on my part. Thanks! – user135827 Nov 5 '09 at 19:11.

Alternatively, if you don't want to monkeypatch File: until my_file. Eof? Do_something_with( my_file.

Read( bytes ) ) end For example, streaming an uploaded tempfile into a new file: # tempfile is a File instance File. Open( new_file, 'wb' ) do |f| # Read in small 65k chunks to limit memory usage f. Write(tempfile.

Read(2**16)) until tempfile. Eof? End.

FILENAME="d:/tmp/file. Bin" class File MEGABYTE = 1024*1024 def each_chunk(chunk_size=MEGABYTE) yield self. Read(chunk_size) until self.

Eof? End end open(FILENAME, "rb") do |f| f. Each_chunk {|chunk| puts chunk } end It works, mbarkhau.

I just moved the constant definition to the File class and added a couple of "self"s for clarity's sake.

1 I wouldn't use the extra constant MEGABYTE, instead: def each_chunk(chunk_size=2**20) – asaaki Jun 3 '11 at 17:12.

I cant really gove you an answer,but what I can give you is a way to a solution, that is you have to find the anglde that you relate to or peaks your interest. A good paper is one that people get drawn into because it reaches them ln some way.As for me WW11 to me, I think of the holocaust and the effect it had on the survivors, their families and those who stood by and did nothing until it was too late.