10 Domain Sockets

5 minute read

Published:

UNIX IPC Summary

Shared Memory

  1. Multiple threads in the same process
    • Process address space shared
    • Need synchro
  2. Multiple related processes
    • Anonymous mapping mmap() with MAP_SHARED | MAP_ANONYMOUS, and then fork()
  3. Multiple unrelated processes
    • mmap() with MAP_SHARED, backed up by a real file with open(), or virtual file with shm_open()

Synchro

  1. Threads
    • Mutex, condition variable, and reader-writer lock
  2. Processes
    • Shared memory required (can be related or unrelated)
      • Unnamed semaphore: sem_init() and sem_destroy()
      • pthread mutex, but initialized with PTHREAD_PROCESS_SHARED attribute in pthread_mutex_init()
    • No shared memory required
      • Pipes (named or unnamed), great for synchro
      • Named semaphore: sem_open(), sem_close(), sem_unlink()
        • Naming requirements…

Passing data

  1. Related processes
    • Unnamed pipe (pipe()), half duplex
  2. Unrelated processes
    • Named pipe, half duplex
  3. Distant processes (network)
    • TCP socket: full duplex, reliable, high overhead, not preserving message boundaries
      • Messages may be chopped or coalesced
    • UDP socket: full duplex, unreliable, low overhead, preserved message boundary

UNIX domain sockets

Different from TCP: two local processes, a bit more powerful (better) than pipes

  • Full duplex
  • Reliable when used in datagram mode (since not on network)
    • Nice because it preserves message boundaries
  • Can transport special things like open file descriptor

Unnamed socket pair

Unnamed pair of connected sockets for related processes

  • created by socketpair(AF_UNIX, ... ) (ana AF_INET)
  • just like a pipe, but full duplex
int socketpair(int domain, int type, int protocol, int sv[2]);

socketpair(AF_UNIX, SOCK_STREAM, 0, fd)

Named

Named local-only socket for unrelated processes

  • created by socket(AF_UNIX, ... )
  • represented by a named special file (ana bind() in IP)
  • Instead of IP/port struct in bind(), pass in file structures
  • Much faster than TCP. Speed like local pipe, but no overhead
    ./recv-unix 3
    ./send-unix ABCDE 8
    

Example

  • Creates named domain socket serv-dom-sock (special file, type s) (ana named pipe)
  • send-unix sends “ABCDE” 8 times to serv-dom-sock
  • recv-unix only grabs the first 3 bytes, and discards the rest
    • Done in datagram mode
    • In TCP, it will continue to read the last 5 bytes, hard :(
  • recv-unix then sends back the bytes to the sender, formatting the bytes a bit. UDP:
    1. Both call bind()
    2. sendto(address), recvfrom()
    • recvfrom() gives client address structure to &clnt_un, so we know it and can send it back

recv-unix.c: (server)

int main(int argc, char **argv) {
	int num_to_recv = atoi(argv[1]);
	int fd;
	
	struct sockaddr_un  un,  clnt_un;  // UDP socket structure
	socklen_t           len, clnt_len;
		
	char *name = "serv-dom-sock";
	
	// create a UNIX domain DATAGRAM socket
	fd = socket(AF_UNIX, SOCK_DGRAM, 0))

	// remove the server domain socket file if exists already
	unlink(name);
	
	// Boiler plate: prepare the server domain socket address structure
	memset(&un, 0, sizeof(un));
	un.sun_family = AF_UNIX;
	strcpy(un.sun_path, name);
	len = offsetof(struct sockaddr_un, sun_path) + strlen(name);
	
	// bind the server domain socket name to the descriptor
	bind(fd, (struct sockaddr *)&un, len) 

	
	char *buf = malloc(num_to_recv + 1);
	char reply[100];
	int i = 0, n;
	
	for (;;) {
		memset(buf, 0, num_to_recv + 1);
		clnt_len = sizeof(clnt_un);
		// recvfrom gives client address structure to &clnt_un
		n = recvfrom(fd, buf, num_to_recv, 0,
					 (struct sockaddr *) &clnt_un, &clnt_len);

		fprintf(stderr, "\"%s\" received from ", buf);
		write(STDERR_FILENO, clnt_un.sun_path,
				clnt_len - offsetof(struct sockaddr_un, sun_path));
		fprintf(stderr, " (clnt_len:%d), ", clnt_len);

		// sends back to client
		snprintf(reply, sizeof(reply), "[%d](%s)", ++i, buf);
		n = sendto(fd, reply, strlen(reply), 0, 
				   (struct sockaddr *) &clnt_un, clnt_len);
		fprintf(stderr, "sent back %d bytes: %s\n", n, reply);
	}
	
	// Clean up
	fprintf(stderr, "Server shutting down\n");
	free(buf);
	close(fd);
	unlink(name);
}
  • send-unix.c: (client)
static void die(const char *m) { perror(m); exit(1); }
	
int main(int argc, char **argv)
{
	const char *msg = argv[1];
	int num_repeat = atoi(argv[2]);
	
	int fd;
	socklen_t my_len;
	struct sockaddr_un my_un;
	
	char my_name[sizeof(my_un.sun_path)];
	snprintf(my_name, sizeof(my_name), "clnt-dom-sock.%d", getpid());
	
	// create a UNIX domain datagram socket
	if ((fd = socket(AF_UNIX, SOCK_DGRAM, 0)) < 0)
		die("socket failed");
	
	// fill in my own domain socket address structure
	memset(&my_un, 0, sizeof(my_un));
	my_un.sun_family = AF_UNIX;
	strcpy(my_un.sun_path, my_name);
	my_len = offsetof(struct sockaddr_un, sun_path) + strlen(my_name);
	
	// bind my own domain sock name to the descriptor
	if (bind(fd, (struct sockaddr *)&my_un, my_len) < 0) 
		die("bind failed");
	
	socklen_t serv_len;
	struct sockaddr_un serv_un;
	char *serv_name = "serv-dom-sock";
	assert(strlen(serv_name) < sizeof(serv_un.sun_path));
	
	// fill in the server's domain socket address structure
	memset(&serv_un, 0, sizeof(serv_un));
	serv_un.sun_family = AF_UNIX;
	strcpy(serv_un.sun_path, serv_name);
	serv_len = offsetof(struct sockaddr_un, sun_path) + strlen(serv_name);
	
	int i, n;
	
	// send messages
	for (i = 0; i < num_repeat; i++) {
		n = sendto(fd, msg, strlen(msg), 0,
				   (struct sockaddr *) &serv_un, serv_len);
		fprintf(stderr, "[%d] sent %d bytes: \"%s\"\n", i+1, n, msg);
	}
	
	// receive replies
	char reply[100];
	for (i = 0; i < num_repeat; i++) {
		memset(reply, 0, sizeof(reply));
		// since bind() called, know address
		n = recvfrom(fd, reply, sizeof(reply), 0, NULL, NULL);
		fprintf(stderr, "%d bytes received: \"%s\"\n", n, reply);
	}
	
	// clean up
	close(fd);
	unlink(my_name);
}

What if send a lot of messages, like 1000?

  • Stuck!!!
    1. Client sends everything, and then goes to a different loop to receive replies
    2. Server receives and sends one-by-one
    3. 1000 message floods the receiving buffer of the server side inside the kernel
    4. sendto() blocking on client side
    5. Server sending messages back, also flooding the client buffer!
    6. Deadlock :( Different behavior across platforms. Not really specified
  • Linux: sendto() mimics pipe (blocks if filled up)
  • Max: sendto() mimics network (fails if filled up)

Passing open fd

Unique: Two table entries from different processes, with different fds, point to the same file table entry!!

alt)

Need to have a (unnamed or named) domain socket

  1. Pass a random open fd over domain socket
  2. Other process receives it
  3. Allocates the next unused fd.