Server that bridges IP/UDP to local MPI communication channels. More...

#include <MpiProxy.h>

Inheritance diagram for MpiProxy:

Data Structures
struct	Header
	Packet payload header for MPI messages via IP/UDP. More...

Public Types
enum	Operation { MpiOpSend = 0, MpiOpRecv, MpiOpExec, MpiOpTerminate }
	Encodes various MPI operations. More...

Public Types inherited from Application
enum	Result { Success, NotFound, IOError, InvalidArgument, ShowUsage, TimedOut, OutOfMemory }
	Result codes. More...

Public Member Functions
	MpiProxy (int argc, char **argv)
	Constructor. More...

virtual	~MpiProxy ()
	Destructor. More...

virtual Result	initialize ()
	Initialize the server. More...

virtual Result	exec ()
	Run the server. More...

Public Member Functions inherited from POSIXApplication
	POSIXApplication (int argc, char **argv)
	Class constructor. More...

virtual	~POSIXApplication ()
	Class destructor. More...

Public Member Functions inherited from Application
	Application (int argc, char **argv)
	Class constructor. More...

virtual	~Application ()
	Class destructor. More...

virtual int	run ()
	Run the application. More...

Static Public Attributes
static const Size	MaximumPacketSize = 1448
	Maximum size of packet payload. More...

Private Member Functions
Result	udpSend (const void *packet, const Size size, const struct sockaddr &addr) const
	Send UDP packet. More...

Result	udpSendMultiple (const struct iovec *vec, const Size count, const struct sockaddr &addr) const
	Send multiple UDP packets. More...

Result	udpReceive (void *packet, Size &size, struct sockaddr &addr) const
	Receive UDP packet. More...

Result	processRequest (const u8 *packet, const Size size, const struct sockaddr &addr)
	Process incoming packet. More...

Result	processSend (const Header header, const u8 packet, const Size size)
	Process MPI send request. More...

Result	processRecv (const Header header, const u8 packet, const Size size, const struct sockaddr &addr)
	Process MPI recv request. More...

Result	processExec (const Header header, const u8 packet, const Size size, const struct sockaddr &addr)
	Process execute request. More...

Result	processTerminate (const Header header, const u8 packet, const Size size, const struct sockaddr &addr)
	Process MPI terminate request. More...

Result	createChannels (const Size rankId, const Size coreCount)
	Create communication channels. More...

Result	startLocalProcess (const char *command, const Size rankId, const Size coreCount)
	Start a process on the local processor. More...

Result	startRemoteProcess (const Size coreId, const char *command, const Size rankId, const Size coreCount)
	Start a process on a secondary processor. More...

Private Attributes
int	m_sock
	IP/UDP socket for external communication. More...

NetworkClient *	m_client
	Networking client object. More...

Memory::Range	m_memChannelBase
	Memory base address for local MPI communication. More...

Index< MemoryChannel, MaximumChannels >	m_readChannels
	Stores all channels for receiving data from processes. More...

Index< MemoryChannel, MaximumChannels >	m_writeChannels
	Stores all channels for sending data to processes. More...

Array< ProcessID, MaximumChannels >	m_pids
	Records the PID of each process participating in the computation. More...

Static Private Attributes
static const u16	UdpPort = 6660
	Port number for IP/UDP traffic. More...

static const Size	ReceiveTimeoutMs = 500
	Timeout in milliseconds to wait for packet receive. More...

static const Size	MaximumChannels = 128u
	Maximum number of supported MPI channels. More...

Additional Inherited Members
Protected Member Functions inherited from POSIXApplication
virtual Result	output (const char *string) const
	Print text to output. More...

int	runProgram (const char path, const char *argv)
	Runs an external program. More...

Protected Member Functions inherited from Application
virtual Result	output (String &string) const
	Print string to output. More...

ArgumentParser &	parser ()
	Get program arguments parser. More...

const ArgumentParser &	parser () const
	Get constant program arguments parser. More...

const ArgumentContainer &	arguments () const
	Get program arguments. More...

void	setVersion (const String &version)
	Set program version. More...

Protected Attributes inherited from Application
int	m_argc
	Input argument count. More...

char **	m_argv
	Input argument values. More...

Detailed Description

Server that bridges IP/UDP to local MPI communication channels.

Warning: This server should only be used on a trusted network. No authentication or encryption supported!

Todo:: This server might be able re-use the MpiTarget class by inheritance or as a member instance

Todo:: MpiProxy protocol currently uses UDP which does not protect against packet loss or corruption. A simple solution can be implemented by using retransmissions and acknowledge packets

Definition at line 50 of file MpiProxy.h.

Member Enumeration Documentation

◆ Operation

enum MpiProxy::Operation

Encodes various MPI operations.

Enumerator
MpiOpSend
MpiOpRecv
MpiOpExec
MpiOpTerminate

Definition at line 71 of file MpiProxy.h.

Constructor & Destructor Documentation

◆ MpiProxy()

MpiProxy::MpiProxy	(	int	argc,
		char **	argv
	)

Constructor.

Parameters

argc	Argument count
argv	Argument values

Definition at line 42 of file MpiProxy.cpp.

References ANY, Sequence< T >::fill(), m_memChannelBase, m_pids, Application::parser(), ArgumentParser::registerPositional(), MemoryBlock::set(), and ArgumentParser::setDescription().

◆ ~MpiProxy()

MpiProxy::~MpiProxy ( )

virtual

Destructor.

Definition at line 54 of file MpiProxy.cpp.

References DEBUG, m_client, and ZERO.

Member Function Documentation

◆ createChannels()

MpiProxy::Result MpiProxy::createChannels	(	const Size	rankId,
		const Size	coreCount
	)

private

Create communication channels.

Parameters

rankId	Rank identifier
coreCount	Total core count in the computation

Returns: Result code

Definition at line 496 of file MpiProxy.cpp.

References Memory::Range::access, Channel::Consumer, DEBUG, ERROR, Index< T, N >::get(), Index< T, N >::insertAt(), Application::IOError, m_memChannelBase, m_readChannels, m_writeChannels, MapContiguous, NOTICE, Application::OutOfMemory, PAGESIZE, Memory::Range::phys, Channel::Producer, Memory::Readable, SELF, MemoryBlock::set(), MemoryChannel::setPhysical(), Memory::Range::size, Channel::Success, Application::Success, API::Success, MemoryChannel::unmap(), Memory::User, Memory::Range::virt, VMCtl(), Memory::Writable, and ZERO.

Referenced by processExec().

◆ exec()

MpiProxy::Result MpiProxy::exec ( )

virtual

Run the server.

Returns: Result code

Implements Application.

Definition at line 109 of file MpiProxy.cpp.

References sockaddr::addr, DEBUG, ERROR, MaximumPacketSize, processRequest(), Application::Success, Application::TimedOut, and udpReceive().

◆ initialize()

MpiProxy::Result MpiProxy::initialize ( )

virtual

Initialize the server.

Returns: Result code

Reimplemented from Application.

Definition at line 64 of file MpiProxy.cpp.

References Application::arguments(), NetworkClient::bindSocket(), NetworkClient::createSocket(), DEBUG, ERROR, ArgumentContainer::get(), NetworkClient::initialize(), Application::IOError, m_client, m_sock, Application::Success, NetworkClient::Success, NetworkClient::UDP, UdpPort, and ZERO.

◆ processExec()

MpiProxy::Result MpiProxy::processExec	(	const Header *	header,
		const u8 *	packet,
		const Size	size,
		const struct sockaddr &	addr
	)

private

Process execute request.

Parameters

header	Packet header pointer
packet	Full packet input
size	Number of bytes received
addr	Source IP and port of the packet

Returns: Result code

Todo:: Check if a computation is already running and reject the request if so

Definition at line 384 of file MpiProxy.cpp.

References MemoryBlock::copy(), MpiProxy::Header::coreId, createChannels(), DEBUG, ERROR, header, FileSystemPath::MaximumLength, MaximumPacketSize, MPI_ERR_IO, MPI_SUCCESS, MpiOpExec, MpiProxy::Header::operation, MpiProxy::Header::rankId, MpiProxy::Header::result, MemoryBlock::set(), startLocalProcess(), startRemoteProcess(), Application::Success, and udpSend().

Referenced by processRequest().

◆ processRecv()

MpiProxy::Result MpiProxy::processRecv	(	const Header *	header,
		const u8 *	packet,
		const Size	size,
		const struct sockaddr &	addr
	)

private

Process MPI recv request.

Parameters

header	Packet header pointer
packet	Full packet input
size	Number of bytes received
addr	Source IP and port of the packet

Returns: Result code

Definition at line 295 of file MpiProxy.cpp.

References assert, ERROR, header, MaximumPacketSize, NetworkQueue::MaxPackets, MPI_INT, MPI_UNSIGNED_CHAR, Application::NotFound, NOTICE, NetworkQueue::PayloadBufferSize, ProcessCtl(), Schedule, and SELF.

Referenced by processRequest().

◆ processRequest()

MpiProxy::Result MpiProxy::processRequest	(	const u8 *	packet,
		const Size	size,
		const struct sockaddr &	addr
	)

private

Process incoming packet.

Parameters

packet	Payload input
size	Number of bytes received
addr	Source IP and port of the packet

Returns: Result code

Definition at line 222 of file MpiProxy.cpp.

References DEBUG, ERROR, Number::Hex, MpiOpExec, MpiOpRecv, MpiOpSend, MpiOpTerminate, MpiProxy::Header::operation, processExec(), processRecv(), processSend(), processTerminate(), and Application::Success.

Referenced by exec().

◆ processSend()

MpiProxy::Result MpiProxy::processSend	(	const Header *	header,
		const u8 *	packet,
		const Size	size
	)

private

Process MPI send request.

Parameters

header	Packet header pointer
packet	Full packet input
size	Number of bytes received

Returns: Result code

Definition at line 252 of file MpiProxy.cpp.

References ERROR, Index< T, N >::get(), header, MPIMessage::integer, m_writeChannels, MPI_INT, MPI_UNSIGNED_CHAR, Application::NotFound, NOTICE, Channel::Success, Application::Success, MPIMessage::uchar, and MemoryChannel::write().

Referenced by processRequest().

◆ processTerminate()

MpiProxy::Result MpiProxy::processTerminate	(	const Header *	header,
		const u8 *	packet,
		const Size	size,
		const struct sockaddr &	addr
	)

private

Process MPI terminate request.

Parameters

header	Packet header pointer
packet	Full packet input
size	Number of bytes received
addr	Source IP and port of the packet

Returns: Result code

Definition at line 435 of file MpiProxy.cpp.

References ANY, MpiProxy::Header::coreId, ERROR, header, Array< T, N >::insert(), Application::IOError, m_memChannelBase, m_pids, MaximumPacketSize, MPI_SUCCESS, MpiOpTerminate, NOTICE, MpiProxy::Header::operation, MpiProxy::Header::rankId, Release, MpiProxy::Header::result, SELF, MemoryBlock::set(), Array< T, N >::size(), Application::Success, API::Success, udpSend(), Memory::Range::virt, VMCtl(), waitpid(), and ZERO.

Referenced by processRequest().

◆ startLocalProcess()

MpiProxy::Result MpiProxy::startLocalProcess	(	const char *	command,
		const Size	rankId,
		const Size	coreCount
	)

private

Start a process on the local processor.

Parameters

command	Command to execute
rankId	Rank identifier
coreCount	Total cores in the computation

Returns: Result code

Definition at line 580 of file MpiProxy.cpp.

References List< T >::count(), ListIterator< T >::current(), DEBUG, Number::Dec, ERROR, ApplicationLauncher::exec(), ApplicationLauncher::getPid(), ListIterator< T >::hasCurrent(), Number::Hex, Array< T, N >::insert(), Application::IOError, m_memChannelBase, m_pids, NOTICE, Memory::Range::phys, String::split(), ApplicationLauncher::Success, Application::Success, and ZERO.

Referenced by processExec().

◆ startRemoteProcess()

MpiProxy::Result MpiProxy::startRemoteProcess	(	const Size	coreId,
		const char *	command,
		const Size	rankId,
		const Size	coreCount
	)

private

Start a process on a secondary processor.

Parameters

coreId	Core identifier of secondary processor
command	Command to execute
rankId	Rank identifier
coreCount	Total cores in the computation

Returns: Result code

Definition at line 642 of file MpiProxy.cpp.

References Memory::Range::access, assert, BufferedFile::buffer(), coreId, CoreClient::createProcess(), ListIterator< T >::current(), DEBUG, Number::Dec, ERROR, Lz4Decompressor::getUncompressedSize(), ListIterator< T >::hasCurrent(), Number::Hex, Lz4Decompressor::initialize(), Array< T, N >::insert(), Application::IOError, m_memChannelBase, m_pids, MapContiguous, Application::NotFound, NULL, Application::OutOfMemory, Memory::Range::phys, BufferedFile::read(), Lz4Decompressor::read(), Memory::Readable, Release, SELF, Memory::Range::size, BufferedFile::size(), String::split(), BufferedFile::Success, Core::Success, Application::Success, API::Success, Lz4Decompressor::Success, Memory::User, Memory::Range::virt, VMCtl(), and Memory::Writable.

Referenced by processExec().

◆ udpReceive()

MpiProxy::Result MpiProxy::udpReceive	(	void *	packet,
		Size &	size,
		struct sockaddr &	addr
	)		const

private

Receive UDP packet.

Parameters

packet	Payload output
size	Output for number of bytes received
addr	Output for the source IP and port

Returns: Result code

Definition at line 185 of file MpiProxy.cpp.

References sockaddr::addr, DEBUG, errno, ERROR, Application::IOError, m_client, m_sock, sockaddr::port, ReceiveTimeoutMs, recvfrom(), strerror(), Application::Success, NetworkClient::Success, Application::TimedOut, NetworkClient::TimedOut, IPV4::toString(), NetworkClient::UDP, and NetworkClient::waitSocket().

Referenced by exec().

◆ udpSend()

MpiProxy::Result MpiProxy::udpSend	(	const void *	packet,
		const Size	size,
		const struct sockaddr &	addr
	)		const

private

Send UDP packet.

Parameters

packet	Payload input
size	Number of bytes to send
addr	The destination IP and port

Returns: Result code

Definition at line 142 of file MpiProxy.cpp.

References sockaddr::addr, DEBUG, errno, ERROR, Application::IOError, m_sock, sendto(), strerror(), and Application::Success.

Referenced by processExec(), and processTerminate().

◆ udpSendMultiple()

MpiProxy::Result MpiProxy::udpSendMultiple	(	const struct iovec *	vec,
		const Size	count,
		const struct sockaddr &	addr
	)		const

private

Send multiple UDP packets.

Parameters

vec	I/O vector with multiple packets
count	Number of entries in the I/O vector
addr	The destination IP and port

Returns: Result code

Definition at line 160 of file MpiProxy.cpp.

References DEBUG, errno, ERROR, Application::IOError, m_sock, msghdr::msg_iov, msghdr::msg_iovlen, msghdr::msg_name, msghdr::msg_namelen, sendmsg(), strerror(), and Application::Success.

Field Documentation

◆ m_client

NetworkClient* MpiProxy::m_client

private

Networking client object.

Definition at line 285 of file MpiProxy.h.

Referenced by initialize(), udpReceive(), and ~MpiProxy().

◆ m_memChannelBase

Memory::Range MpiProxy::m_memChannelBase

private

Memory base address for local MPI communication.

Definition at line 288 of file MpiProxy.h.

Referenced by createChannels(), MpiProxy(), processTerminate(), startLocalProcess(), and startRemoteProcess().

◆ m_pids

Array<ProcessID, MaximumChannels> MpiProxy::m_pids

private

Records the PID of each process participating in the computation.

Definition at line 297 of file MpiProxy.h.

Referenced by MpiProxy(), processTerminate(), startLocalProcess(), and startRemoteProcess().

◆ m_readChannels

Index<MemoryChannel, MaximumChannels> MpiProxy::m_readChannels

private

Stores all channels for receiving data from processes.

Definition at line 291 of file MpiProxy.h.

Referenced by createChannels().

◆ m_sock

int MpiProxy::m_sock

private

IP/UDP socket for external communication.

Definition at line 282 of file MpiProxy.h.

Referenced by initialize(), udpReceive(), udpSend(), and udpSendMultiple().

◆ m_writeChannels

Index<MemoryChannel, MaximumChannels> MpiProxy::m_writeChannels

private

Stores all channels for sending data to processes.

Definition at line 294 of file MpiProxy.h.

Referenced by createChannels(), and processSend().

◆ MaximumChannels

const Size MpiProxy::MaximumChannels = 128u

staticprivate

Maximum number of supported MPI channels.

Definition at line 61 of file MpiProxy.h.

◆ MaximumPacketSize

const Size MpiProxy::MaximumPacketSize = 1448

static

Maximum size of packet payload.

Definition at line 66 of file MpiProxy.h.

Referenced by exec(), processExec(), processRecv(), processTerminate(), MpiHost::receive(), MpiHost::send(), MpiHost::startProcesses(), and MpiHost::terminate().

◆ ReceiveTimeoutMs

const Size MpiProxy::ReceiveTimeoutMs = 500

staticprivate

Timeout in milliseconds to wait for packet receive.

Definition at line 58 of file MpiProxy.h.

Referenced by udpReceive().

◆ UdpPort

const u16 MpiProxy::UdpPort = 6660

staticprivate

Port number for IP/UDP traffic.

Definition at line 55 of file MpiProxy.h.

Referenced by initialize().

The documentation for this class was generated from the following files:

server/mpiproxy/MpiProxy.h
server/mpiproxy/MpiProxy.cpp

Data Structures

Public Types

Public Member Functions

Static Public Attributes

Private Member Functions

Private Attributes

Static Private Attributes

Additional Inherited Members

Detailed Description

Member Enumeration Documentation

◆ Operation

Constructor & Destructor Documentation

◆ MpiProxy()

◆ ~MpiProxy()

Member Function Documentation

◆ createChannels()

◆ exec()

◆ initialize()

◆ processExec()

◆ processRecv()

◆ processRequest()

◆ processSend()

◆ processTerminate()

◆ startLocalProcess()

◆ startRemoteProcess()

◆ udpReceive()

◆ udpSend()

◆ udpSendMultiple()

Field Documentation

◆ m_client

◆ m_memChannelBase

◆ m_pids

◆ m_readChannels

◆ m_sock

◆ m_writeChannels

◆ MaximumChannels

◆ MaximumPacketSize

◆ ReceiveTimeoutMs

◆ UdpPort