How to create a file in RAM ?

[kathy]

Read / Write to disk file is too slow. How to create a file in RAM
memory so that I could read / write to memory?

[kathy]

On Aug 7, 9:10 am, kathy <yqin...@yahoo.com> wrote:
> Read / Write to disk file is too slow. How to create a file in RAM
> memory so that I could read / write to memory?

by the way, I would like to read line by line. My file is text file.

[Tom Serface]

If you are only doing text why not use a vector or CStringArray and just
manipulate the lines of text there and then eventually write it to a file?

Tom

"kathy" <yqi...@yahoo.com> wrote in message
news:004aeaba-eb8a-4a4a...@n2g2000vba.googlegroups.com...

[kathy]

On Aug 7, 10:37 am, "Tom Serface" <t...@camaswood.com> wrote:
> If you are only doing text why not use a vector or CStringArray and just
> manipulate the lines of text there and then eventually write it to a file?
>
> Tom

Yes. I am using string vector. But write / read from vector to file
are tedious - have to search the"\n"? How to read/write string vector
to CMemFile?

[Tom Serface]

If you are using MFC, you could use a CStringArray (although I think vector
is similar). Reading and writing to a file would be a simple algorithm.
You shouldn't have to pay attention to the new lines at all. When you want
to read you just read each line using CStdioFile::ReadString() and add it to
the vector or CStringArray (in file order), then to write out just loop
through the vector or CStringArray and using CStdioFile::WriteString() to
write out the persistent file.

I don't know of a CStdioMemFile sort of thing (frankly I think that would be
almost the same as a CStringArray or vector without any structure). Even
so, you'd have to do the work of looking for the end of lines unless you
already had the lines segregated (like in an array or vector).

I think if you use CStdioFile::ReadString and CStdioFile::WriteString you
will find it easier to translate from vector to persistent file.

Tom

"kathy" <yqi...@yahoo.com> wrote in message

news:6586f165-c1fc-4d99...@g6g2000vbr.googlegroups.com...

[Giovanni Dicanio]

Tom Serface ha scritto:

> I think if you use CStdioFile::ReadString and CStdioFile::WriteString
> you will find it easier to translate from vector to persistent file.

I'm not sure about the exact kind of text (UTF-8, UTF-16, plain
ASCII...) that the OP is going to read from/store to files, but I think
that CStdioFileEx class freely available on CodeProject is a better
choice than MFC's built-in CStdioFile class:

http://www.codeproject.com/KB/files/stdiofileex.aspx

Giovanni

[Giovanni Dicanio]

Tom Serface ha scritto:

> If you are using MFC, you could use a CStringArray (although I think
> vector is similar).

If the file to be read contains lots of lines, and the line count is
unknown, probably the std::vector 1.5X growing algorithm offers better
performance than CStringArray arithmetic growing algorithm.

> Reading and writing to a file would be a simple
> algorithm. You shouldn't have to pay attention to the new lines at all.
> When you want to read you just read each line using
> CStdioFile::ReadString() and add it to the vector or CStringArray (in
> file order), then to write out just loop through the vector or
> CStringArray and using CStdioFile::WriteString() to write out the
> persistent file.

Another option might be to read the whole file in RAM in one single
operation, and then do the parsing in RAM.

e.g.

- get the file size
- allocate a block of heap memory to store the file (using 'new', or
just using std::vector)
- read the whole file in that memory block
- close the file
- scan the memory buffer, using '\n' as line separator

Giovanni

[Giovanni Dicanio]

kathy ha scritto:

In addition to what we already wrote, you may want to use memory mapped
techniques to access file for reading.

You can find a complete MFC simple application (with source code) that
reads a text file using memory mapping and parses file lines here:

http://www.geocities.com/giovanni.dicanio/vc/TestFileInMemory.zip

The class that implements the simple memory mapping technique for file
reading is CMemoryMappedTextFile; instead you can find the line parsing
algorithm in CTestReadFileDlg::OnBnClickedReadFile() method.

The content of the file is assumed to be plain ASCII (no Unicode).
In case you have e.g. a UTF-8 text file, you can simply convert the read
lines to Unicode UTF-16 using ::MultiByteToWideChar Win32 API.

Here is part of the code that uses the read-only memory-mapped text file
class to parse file content line by line.
(This code is from CTestReadFileDlg::OnBnClickedReadFile() method.)

<code>

...

// Clear displayed file content
m_lstFileContent.ResetContent();

// Open file for reading
CMemoryMappedTextFile file(filename);
if (! file.IsReady())
{
AfxMessageBox(IDS_ERROR_FILE_OPEN, MB_OK|MB_ICONERROR);
return;
}

//
// Parse file lines from memory mapped file.
// Assume that the file stores pure ASCII data.
//

// Points to the beginning of file.
// This pointer will be moved forward during file scanning.
const char * pchFileContent = file.GetBuffer();
ASSERT(pchFileContent != NULL);

// Pointer to end of file
const char * pchEnd = pchFileContent + file.GetLength();

// Parse file content, adding each line to the listbox
while (pchFileContent < pchEnd)
{
// Find line terminator ('\n')
const char * pchEndOfLine = std::find(pchFileContent, pchEnd,
'\n');

// Get length of line in characters
int lineLength = pchEndOfLine - pchFileContent - 1;

// Store a copy of current line using CStringA instance
// (text is not Unicode)
CStringA line(pchFileContent, lineLength );

// Move scanning pointer to next line
pchFileContent = pchEndOfLine + 1;

// Add string to listbox
m_lstFileContent.AddString( CA2T(line) );
}

// CMemoryMappedTextFile destructor does proper cleanup.
// (Thanks RAII)

</code>

The implementation of CMemoryMappedTextFile class follows:

--------------------------------------------------------------------------
--------------------------------------------------------------------------

<file name="MemoryMappedTextFile.h">

//////////////////////////////////////////////////////////////////////////
// FILE: MemoryMappedTextFile.h
//////////////////////////////////////////////////////////////////////////

#pragma once

//========================================================================
// Manages a read-only memory-mapped file
// (maximum 4 GB in size).
//========================================================================
class CMemoryMappedTextFile
{
public:

// Initializes the object to read the specified file
CMemoryMappedTextFile(LPCTSTR filename);

// Cleanup
~CMemoryMappedTextFile();

// Pointer to file content
// (if object initialization failed, NULL is returned)
const char * GetBuffer() const;

// Returns file size, in bytes
DWORD GetLength() const;

// Was file mapping created successfully?
BOOL IsReady() const;

//
// IMPLEMENTATION
//
private:
HANDLE m_hFile; // File handle
HANDLE m_hFileMapping; // File mapping handle
char * m_pContent; // Pointer to file content
DWORD m_cbSize; // File size, in bytes
};

//------------------------------------------------------------------------
// INLINE METHOD IMPLEMENTATIONS
//------------------------------------------------------------------------

inline const char * CMemoryMappedTextFile::GetBuffer() const
{
return m_pContent;
}

inline DWORD CMemoryMappedTextFile::GetLength() const
{
return m_cbSize;
}

inline BOOL CMemoryMappedTextFile::IsReady() const
{
return ( m_pContent != NULL ? TRUE : FALSE );
}

</file>

--------------------------------------------------------------------------
--------------------------------------------------------------------------

<file name="MemoryMappedTextFile.cpp">

//////////////////////////////////////////////////////////////////////////
// FILE: MemoryMappedTextFile.cpp
//////////////////////////////////////////////////////////////////////////

#include "stdafx.h" // Precompiled header
#include "MemoryMappedTextFile.h" // Class header

//------------------------------------------------------------------------
// METHOD IMPLEMENTATIONS
//------------------------------------------------------------------------

CMemoryMappedTextFile::CMemoryMappedTextFile(LPCTSTR filename)
: m_hFile(INVALID_HANDLE_VALUE), m_hFileMapping(NULL), m_pContent(NULL),
m_cbSize(0)
{
ASSERT(filename != NULL);

// Open file for reading
m_hFile = CreateFile(
filename,
GENERIC_READ,
FILE_SHARE_READ,
NULL,
OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL,
NULL
);
if (m_hFile == INVALID_HANDLE_VALUE)
return;

// Get file size
DWORD cbSize = GetFileSize(m_hFile, NULL);

// Create file mapping (read-only)
m_hFileMapping = CreateFileMapping(
m_hFile,
NULL,
PAGE_READONLY,
0,
0,
NULL
);
if (m_hFileMapping == NULL)
return;

// Get pointer to file content
m_pContent = reinterpret_cast<PCHAR>(
MapViewOfFile(m_hFileMapping, FILE_MAP_READ, 0, 0, cbSize)
);
if (m_pContent == NULL)
return;

// Store size field
m_cbSize = cbSize;

}

CMemoryMappedTextFile::~CMemoryMappedTextFile()
{
//
// Safely cleanup everything
//

if (m_pContent)
{
UnmapViewOfFile(m_pContent);
m_pContent = NULL;
}

if (m_hFileMapping)
{
CloseHandle(m_hFileMapping);
m_hFileMapping = NULL;
}

if (m_hFile != INVALID_HANDLE_VALUE)
{
CloseHandle(m_hFile);
m_hFile = INVALID_HANDLE_VALUE;
}
}

</file>

--------------------------------------------------------------------------
--------------------------------------------------------------------------

HTH,
Giovanni

[Tom Serface]

True, but that doesn't affect the idea of using a vector or array to store
and manipulate the lines in memory :o)

Tom

"Giovanni Dicanio" <giovanniD...@REMOVEMEgmail.com> wrote in message
news:OOT0iLRG...@TK2MSFTNGP02.phx.gbl...

[Tom Serface]

I agree that std:vector is likely faster than CStringArray. Certainly some
testing should be done to determine the best course.

Tom

"Giovanni Dicanio" <giovanniD...@REMOVEMEgmail.com> wrote in message

news:eW0kbQRG...@TK2MSFTNGP02.phx.gbl...