Playing with shared memory - problem with non 'simple' data types

[winbatch]

Hi,

In my never ending quest to improve my C++, I'm trying to write a class that sets and reads shared memory. For the time being I am avoiding dealing with issues of concurrency/locking, etc. and starting small. I was successfully able to put into shared memory things like integers, char *, and even my own custom classes. (By making it a templated class).

However, I seem to be running into a problem if I attempt to use either a datatype or a class that contains a member that is of a datatype 'non native', ie like a 'string', or 'vector', I either crash or get garbage. I'm coming to the conclusion that I simply can't store these datatypes, given that the shared memory is 'C' code rather than C++. Could you confirm my assumption that it basically can't handle C++ types, or is it something in my code that would cause this problem?

Below is my SharedMemory class. (Note that all code is in the .h file right now - I'll separate it out when I use it more officially).

extern "C"
{
 #include <sys/types.h>
 #include <sys/ipc.h>
 #include <sys/shm.h>
}
template <class InputClass>
class SharedMemory
{
 private:
  key_t key;
  int shmid;
  InputClass *data;
  int SHM_SIZE;
  string fileName;
  
  void initialize()
  {
   if ((key = ftok(fileName.c_str(), 'R')) == -1) 
   {
	   throw DHException( "*** Could not connect to shared memory file [" + fileName + "] Check that the file exists...***" );
 }
  
  /* connect to (and possibly create) the segment: */
	  if ((shmid = shmget(key, SHM_SIZE, 0644 | IPC_CREAT)) == -1)
	  {
		throw DHException( "*** Could not use shared memory file: [" + fileName + "] Check that the allocated size is OK..***");
	  }
	  
	  data = (InputClass *) shmat(shmid, 0, 0);
	  if (data == (InputClass *)(-1)) 
	  {
	   throw DHException ( "*** Could not connect to shared memory file:(error 3):" + fileName);
	  }
  }
  
  public:
  SharedMemory( string file )
  {
   SHM_SIZE=sizeof(InputClass);
   fileName = file;
   initialize();
   
  }
  
  SharedMemory( int size, string file )
  {
   SHM_SIZE=size;
   fileName = file;
   initialize();
  }
  
  
  void writeToSharedMemory( InputClass ic )
  {
   *data = ic;
  }
  
  InputClass getFromSharedMemory( )
  {
   return *data;
  }
  
  ~SharedMemory()
  {
   
   if (shmdt((char *)data) == -1) 
   {
	throw DHException ( "Could not disconnect from shared memory file:" + fileName );
	}
  }
  
  int getMemorySize() { return SHM_SIZE; }

An example of usage would be (where Date is a custom class of mine that only contains native types like int's, time_t, etc.):

SharedMemory<Date> shm(1000, "DANTEST" );
 
 while ( true )
 {
  
  Date a;
  shm.writeToSharedMemory(a);
  screenLog<<"Just Placed:"<<a<<endl;
	
  sleep( 1 );
  
 }

[Dave Sinkula]

I'm not familiar with the libraries you are working with, but perhaps the "'non native', ie like a 'string', or 'vector'" issue is because they are not POD types?

[winbatch]

I think my made up definition of 'non native' is the same as your 'non POD'. With that having been said, are they not useable with shared memory?

[Dave Sinkula]

Well I think I had plain old data confused with "things that need a deep copy; things for which a shallow copy would not suffice" -- that is, things that have dynamically-allocated memory rather than a simple array. Something like that.

[winbatch]

Unfortunately, that went over my head. (Was that a no, you can't use dynamically allocated datatypes, or yes, you can)

[Dave Sinkula]

First, I looked up a better description of POD. (This seems to say "yup" to your previous post.)

Unfortunately, that went over my head. (Was that a no, you can't use dynamically allocated datatypes, or yes, you can)

And I found a better description of deep copy. But essentially, I believe I was saying no. I'll qualify that with: perhaps you may, but you'd need to make a deep copy.[?]

[winbatch]

That's interesting because I found that if I was reading and writing from the shared memory inside the same program it worked, but when trying to read those special parts (string, vector) from another program, that's when it failed. Could be because of the shallow/deep copy. I guess I'll have to look into how to deal with that. (Though if I'm simply using the std::string, I don't think it should be me who creates a proper copy constructor for it?)

[Dave Sinkula]

That's interesting because I found that if I was reading and writing from the shared memory inside the same program it worked, but when trying to read those special parts (string, vector) from another program, that's when it failed.

I'm venturing forth into the unknown a bit here, but I'll offer a guess. Here is some code that probably does things it shouldn't, but I'm using it as a prop for my guesses. (Doesn't that make it sound great.) :rolleyes:

#include <iostream>
#include <string>
#include <cstddef>

void showobject(const void *object, std::size_t size)
{
   const unsigned char *byte = static_cast<const unsigned char *>(object);
   for (size_t i = 0; i < size; ++i)
   {
      std::cout << std::hex << static_cast<int>(byte[i]);
   }
   std::cout << std::endl;
}

int main()
{
   std::string text("hello world");
   std::cout << text << " : ";
   showobject(static_cast<const void *>(&text), sizeof text);
   std::cout << text.c_str() << " : ";
   showobject(static_cast<const void *>(text.c_str()), text.size());
   std::cout << static_cast<const void *>(&text[0]) << std::endl;
   return 0;
}

/* my output
hello world : 4454201000143b7b01000
hello world : 68656c6c6f20776f726c64
007B3B14
007B3B14
*/

You can kinda see that something in the string points to the memory that contains the actual string text.

Let's say that Program A and Program B share a string using shared memory. That means each shares the whole first line (of the output displayed). But Program A's memory is different from Program B's, so Program A points to a "hello world", but Program B's memory at 007B3B14 is different.

Anybody feel free to bail me out if I'm completely off target here.

[Dogtree]

The diagnosis is accurate for the most part. Shared memory can't handle non-POD types. You could write a special purpose allocator for your standard containers, and unshared proxy objects that refer to a reasonable representation of the data in shared memory for everything else. I don't have a lot of experience with shared memory, so I won't try to give you code because it would be hopelessly broken. But this is a good problem to solve with threads rather than IPC and processes. ;)

I imagine that the proxy solution would be the easiest for you. You can serialize the data into, say, a C-style string representation for storage in shared memory, and the proxy class would be able to represent it as whatever object you need when you need it for a minor conversion cost. Here's a crude example for std::string:

class string_proxy {
  std::string _str;
  char *_mem;
public:
  std::string_proxy(char *addr): _mem(addr) {}
  std::string& get() { _str.assign(_mem); }
  void serialize()
  {
    memcpy(_mem, _str.c_str(), _str.size() + 1);
  }
};

With std::string and other standard containers you can write a custom allocator that knows about the shared memory and doesn't make your system cry when you use it. But that's harder, and depends heavily on knowing what you're doing with the shared memory library, which I don't, so I won't show you code. ;)

[winbatch]

To make sure I understand, basically you write a function that takes your class contents, makes in effect a big string out of it. It's that string I put in shared memory. You then have a function that can take the big string and turn it back into the class contents again. I can then read the big string from shared memory and then reform the class.

If I've got you right, as long as I make my shared memory segment large enough to handle the size the string version I should be ok. Is that right?

[Dogtree]

Basically, yea. :) But you don't have to use a string representation, you can use whatever you want as long as it's POD.