MUSCLE

MUSCLE stands for Multi User Server Client Linkage Environment and it’s an open source, free, robust, scalable and cross-platform messaging system that comprises of server and client components. It will help you to write all sorts of distributed software, from multiplayer games and IRC chat clients, to SETI calculation apps.

Features at a glance

With MUSCLE, users will be able to easily customize the included "muscled server" by defining their own session logic or message-streaming protocol. The software helps you to write your client code in either of the C++, C, C#, Java, Delphi or Python programming languages.

The program uses BMessage-like muscle::Messages, supports sending of flattened muscle::Messages between computers via TCP (Transmission Control Protocol) streams, deploys a “muscled server” on a server machine, allowing you to write client programs that can be used to communicate with the server.

Additionally, it includes single- and multi-threaded messaging APIs that support all the aforementioned programming languages. The “muscled server” can be easily customized by defining a message-streaming protocol or a session logic.

Getting started with MUSCLE

After a successful login, the client programs can determine who else is logged into the central server, as well as to store muscle::Messages in the RAM (system memory) and send muscle::Messages to other clients that are connected to the server. Please note that client programs can subscribe to specific data on the server, and get instant notifications when the data changes.

Supports a wide range of message routings

The software has been designed to supports a wide range of message routings, including unicast-style, broadcast-style and multicast-style ones through an intelligent, hierarchical pattern-matching routing mechanism.

Supported OSes and availability

MUSCLE runs on any POSIX-compliant operating system. It is distributed as a single, universal sources archive that lets users to optimize the application for their GNU/Linux system. Both 32-bit and 64-bit hardware platforms are supported at this time.

What is new in this release:

• tests/Makefile-mt no longer tries to compile testreflectsession except when executing on an OS that testreflectsession supports.
• Did some minor editing and re-arranging of the "MUSCLE by Example" pages.
• Replaced a number of `tags` in the MkDocs with [URLs](...)
• Removed private-inheritance of CountedObject from all the classes that had it -- now they include a CountedObject private member variable instead. This was done only because all that private inheritance was cluttering up the DOxygen inheritance graphs.
• Added a DECLARE_COUNTED_OBJECT macro to CountedObject.h so that any potential memory-overhead of CountedObjects can be eliminated in cases where object-counting is not desired.
• Removed support for the -DMUSCLE_AVOID_OBJECT_COUNTING and added a -DMUSCLE_ENABLE_OBJECT_COUNTING macro in its place. (That is, the CountedObject class is now disabled-by-default and must be explicitly enabled in order to use it)
• Added "MUSCLE by Example" documentation for MiscUtilityFunctions.{cpp,h}
• Changed a number of `tags` in the MUSCLE-by-Example MkDocs source into [urls] so that the methods they mention can be quickly reviewed.
• server/Makefile now specifies libmuscle.a after the main() .o file.
• Fixed some gcc warnings in the tests folder (per Mika's suggestions)
• UnparseFile() wasn't quoting keywords with spaces properly. Fixed.
• Fixed the dependencies in the "MUSCLE by Example" examples' Makefiles.

What is new in version :

• Added a ShrinkToFit() convenience method to the Queue, Hashtable, and String classes. This method shrinks the object so that the amount of memory it has allocated internally matches the size of the data it is actually holding.
• Hashtable::EnsureSize() and Queue::EnsureSize() now take an optional (allowShrink) argument, that (if set to true) allows the object's internally-allocated array to be reallocated smaller if it is larger than necessary.
• Tweaked the buffer-expansion behavior of the String class to be a bit more efficient.
• Added a guard against a potential infinite recursion that could occur while logging an "OUT OF MEMORY" error after a memory allocation failure, if a LogCallback tried to allocate memory.

What is new in version 6.01:

• Added a PrependWord() convenience method to the String class.
• Added WithReplacements() convenience methods to the String class.
• Added a SetExplicitDelayMicros() method to the DetectNetworkConfigChangesSession class.
• Added a IsCopperDetected() method to the NetworkInterfaceInfo class, so that code can tell whether or not a Ethernet jack has a cable plugged in to it.
• Added a "quietsend" argument to hexterm.
• The NetworkInterfacesChanged() virtual method in the DetectNetworkConfigChangesSession class has been changed to take an argument that calls out which network interfaces in particular have changed. This functionality is currently only implemented under Linux, MacOS/X and Windows. For other OS's the argument will always be an empty list.
• Fixed a bug in the Linux implementation of DetectNetworkConfigChangesSession that could cause a segmentation fault if recvmsg() returned an error (e.g. due to a signal being received).

What is new in version 6.00:

• Rewrote the SSLSocketDataIO class to work better with non-blocking I/O (in conjunction with the new SSLSocketAdapterGateway class).
• Added implementations of SSLSocketDataIO::SetPrivateKey() and SSLSocketDataIO::SetCertificate() that take a ByteBuffer as an argument.
• Added an SSLSocketAdapterGateway class that is used to manage OpenSSL's internal state machine when using an SSLSocketDataIO class with your gateway.
• Added SetSSLPrivateKey() and GetSSLPrivateKey() methods to the ReflectServer class, for easier enabling of SSL authentication on all incoming TCP connections. These methods are available iff MUSCLE_ENABLE_SSL is defined.
• Added SetSSLPublicKeyCertificate() and GetSSLPublicKeyCertificate() methods to the ReflectServer class, for easier enabling of SSL authentication on outgoing TCP connections. These methods are available iff MUSCLE_ENABLE_SSL is defined.
• Added SetSSLPrivateKey() and SetSSLPublicKeyCertificate() methods to the MessageTransceiverThread class, for easier enabling of SSL functionality when using threaded I/O.
• Added an ssl_data folder with some info on generating OpenSSL public/private keypairs, and an example keypair for use in testing OpenSSL.
• When MUSCLE_ENABLE_SSL is defined, muscled now accepts an optional 'privatekey=filename' argument. When specified, SSL mode will be enabled and muscled will only accept incoming TCP connections that present public keys that match this private key/certificate.
• When MUSCLE_ENABLE_SSL is defined, portablereflectclient and qt_example will now accept an optional 'publickey=filename' argument. When specified, SSL mode will be enabled and these clients will connect to muscled using OpenSSL and present this file as their credentials.
• Added an "Animate" checkbox to the qt_example demo. Checking it causes the window to move its indicator around automatically. This is fun and also useful if you want to test a scenario where multiple clients are generating traffic simultaneously.
• Made the qt_example demo prettier.
• Renamed the C++11-helper macros in Hashtable.h and Queue.h to make them less likely to collide with other packages' macros. * Fixed some minor errors in the SSLSocketDataIO class. o Renamed SSLSocketDataIO::SetKey() to SetPrivateKey(). o Renamed SSLSocketDataIO::SetCertificate() to SetPublicKeyCertificate(). o AbstractMessageIOGateway::SetDataIO() is now a virtual method.

What is new in version 5.92:

• Improved support for C++11 move-semantics in the Queue and Hashtable classes (enabled only when -DMUSCLE_USE_CPLUSPLUS11 is defined)
• Added some instrumentation to the String class so I can watch how many times a String object is copied, moved, etc (enabled only when -DMUSCLE_COUNT_STRING_COPY_OPERATIONS is defined)
• Added a PrintAndClearStringCopyCounts() function which will print out the String-operation data collected above.
• Added some SFINAE magic to muscleSwap() so that it will swap by calling SwapContents() when possible, rather than by copying to a temporary object.
• Added an initializer-list constructor and overload of AddTailMulti() to the Queue class (available only when -DMUSCLE_USE_CPLUSPLUS11 is defined, of course)
• Renamed the Queue and array overloads of Queue::AddTail() to AddTailMulti(), to avoid conflicts with the new C++11 template parsing support.
• Renamed the Queue and array overloads of Queue::AddHead() to AddHeadMulti(), to avoid conflicts with the new C++11 template parsing support.
• Replaced MCRASH_IMPL macro with a call to assert(false).
• A little more Android-compatibility tweakage.
• Many of the programs in the tests folder weren't compiling under C++11. Fixed.
• Fixed several potential bugs that were detected by clang's static analysis tool.

What is new in version 5.91:

• Added EnsureCanPut() convenience method to the Hashtable class.
• Added EnsureCanAdd() convenience method to the Queue class.
• Changed DoMutexAtomicIncrement() to be an inline function to make calling it more efficient.
• Changed QMessageTransceiverThread and QAcceptSocketsThread to call QCoreApplication::postEvent() rather than QApplication::postEvent(), to allow for non-GUI Qt apps.
• Updated the Beginner's Guide document to reflect MUSCLE's improved UDP support.
• Merged in some Android compatibility changes provided by Jean-FranA§ois Mullet.
• Use of the MUSCLE_USE_MUTEXES_FOR_ATOMIC_OPERATIONS compile flag would cause MUSCLE to crash on startup due to an order-of-operations issue. This has been fixed now.
• The MUSCLE_USE_MUTEXES_FOR_ATOMIC_OPERATIONS compile flag was previously only used if no other implementation of AtomicCounter was available. Now the flag has higher precedence, so setting the flag means that Mutexes will be used, even if another (more efficient) mechanism is available.

What is new in version 5.90:

• Added a GetPacketMaximumSize() method to the DataIO class to allow gateway code to more intelligently handle UDP-style packetized communication.
• MessageIOGateway now works usefully in conjunction with UDPSocketDataIO.
• Added CreateObjectFromArchiveMessage() templated functions to Message.h, to serve as a restoring-side counterpart to GetArchiveMessageFromPool(), etc.
• AtomicCounter::AtomicIncrement() now returns a boolean (true iff the new counter value is equal to one).
• Modified the HashtableIterator class so that read-only Hashtable iterations are now thread-safe even if the HTIT_FLAG_NOREGISTER flag isn't specified.
• Added a muscle_thread_id class to SetupSystem.h, to properly represent a thread ID in an implementation-neutral fashion.
• Added a "deadlock" program to the tests folder. This program deliberately risks creating a deadlock, as a way to exercise/demonstrate the deadlockfinder test.
• Added support for a -DMUSCLE_AVOID_THREAD_SAFE_HASHTABLE_ITERATORS command line flag, for those who would rather avoid the overhead of automatic thread safety and promise to supply HTIT_FLAG_NOREGISTER arguments by hand where necessary.
• Added an optional LRU lookup cache to the GetHostByName() function, so that it can return more quickly when the same hostnames are getting resolved over and over again.
• Added a SetHostNameCacheSettings() function that enables and adjusts the LRU lookup cache in GetHostByName().
• Added support for "dnscache" and "dnscachesize" command line arguments in HandleStandardDaemonArgs(), to allow command-line specification of the LRU lookup cache's behavior.
• Modified the Hashtable class so that the _iterHead, _iterTail, and _freeHead member values are now uint32s rather than pointers, to reduce memory usage.
• Removed the ThreadLocalStorage::SetFreeHeldObjectsOnExit() method, and added a boolean argument to its constructor instead, since pthreads don't allow you to change that setting after pthread_key_create() has been called.
• Moved GetCurrentThreadID() into the muscle_thread_id class as a static member function, and changed it to return a muscle_thread_id object rather than unsigned long.
• Changed the default hostname for sessions without a known IP address from "" to "_unknown_", as the angle brackets in the former string have a special meaning as of the 5.84 release, and that could interfere with node-path matching in unintended ways.
• The CalculateChecksum() methods in Message.cpp have been modified to be more robust in detecting data transposition differences.
• Removed the MUSCLE_USE_QT_FOR_ATOMIC_OPERATIONS support from AtomicCounter.h, since Qt's QAtomicInt class doesn't support the functionality that the AtomicIncrement() method's new return value requires.
• Removed MessageIOGateway::FlattenMessage() and MessageIOGateway::UnflattenMessage(). Added in their place: MessageIOGateway::FlattenHeaderAndMessage() and MessageIOGateway::UnflattenHeaderAndMessage(). These new methods deal with both the header bytes and the Message body at the same time.
• Added a udpproxy.vcproj file to the tests folder, to help compile udpproxy under Windows.
• Message:Flatten() now iterates over the fields in the Message once, instead of twice.
• GetCurrentThreadID() is now an inline function, since it may now get called often by HashtableIterator.
• Modified the deadlockfinder code to use Queues instead of Hashtables, since muscle_thread_id can't be used as a Hashtable key type anymore.
• Fixed testudp.cpp to properly use a MessageIOGateway for its UDP communication.
• Tweaked the ifdefs in FilePathInfo.cpp a bit more so that statInfo.st_birthtimespec won't be accessed when using MacOS/X SDKs that don't provide it.
• MessageDataIOGateway no longer tries to Unflatten a Message from a zlib-deflated data buffer that it was unable re-inflate.
• Fixed a bug in SendDataUDP() that could cause SendDataUDP() to incorrectly return an error when sending to a multicast address using non-blocking mode, and the output buffer was full.

What is new in version 5.85:

• Added LogTime(MUSCLE_LOG_DEBUG) calls to all the error paths in MessageIOGateway::DoInputImplementation() and Message::Unflatten(), so that it's easier to determine when TCP connections are being aborted due to data corruption.
• Added a PreviousOperationHadTransientFailure() function, which returns true iff errno is EINTR or ENOBUFS.
• Specifying spamspersecond=-1 will now cause hexterm to send spam data as fast as possible.
• SocketMultiplexer.h's MUSCLE_USE_POLL implementation was supplying POLLERR to WSAPoll() but WSAPoll() doesn't support POLLERR so WSAPoll() would return an error when this occurred. Worked around the problem by filtering out POLLERR when compiling under Windows.
• Fixed a bug where send() returning ENOBUFS could cause the socket connection to be terminated, even though ENOBUFS is not a fatal condition.
• SocketMultiplexer.cpp would not compile when MUSCLE_USE_POLL was defined. Fixed.
• The ZLibCodec::Deflate() method would fail to compress all the data in a very large buffer (e.g. over 42MB). Fixed.

What is new in version 5.84:

• The StringMatcher class's numeric-range syntax has been extended so that you can now specify multiple ranges. For example, "" would match strings "19", "20", "21", "25", "30", "31", [...], and "50".
• Added GetCurrentTime64ForRunTime64() and GetRunTime64ForCurrenTime64() conversion functions to TimeUtilityFunctions.h.
• Added a GetDescendant() utility method to the DataNode class.
• Added C++11 move-constructors and move-assignment-operators to the Hashtable, Queue, String, Message, and ByteBuffer classes. For backwards compatibility with older compilers, this code will only be compiled if -DMUSCLE_USE_CPLUSPLUS11 is specified on the compile line.
• SharedMemory class will now nerf itself into a non-shared-memory class if -DMUSCLE_FAKE_SHARED_MEMORY is specified.
• Added a testfilepathinfo test to the tests folder.
• Updated all copyright notice headers to read 2000-2013 Meyer Sound.
• Added spaces between macro tokens (e.g. UINT32_FORMAT_SPEC) and string constants (e.g. "Hello") to make C++11 compilers happy.
• ByteBuffer.cpp had a syntax error that would prevent it from compiling on big-endian hosts. Fixed.
• MacOS/X only: Replaced deprecated Carbon function calls with Mach equivalents, to avoid deprecation warnings under 10.8.x.

What is new in version 5.83:

• Added convenience versions of InflateByteBuffer() and DeflateByteBuffer() that take a ByteBufferRef as an argument.
• Removed some obsolete/unused methods (EnsureBufferSize() and FreeLargeBuffer()) from the AbstractMessageIOGateway class.
• Fixed some typos in the comments in the delphi subfolder.
• The Hashtable class no longer generates warnings when compiled under MSVC with -DMUSCLE_AVOID_MINIMIZED_HASHTABLES defined.
• Fixed a bug in IPAddressAndPort::ToString() that caused IPv4 address strings to be formatted ambiguously when (preferIPv4Style) was set to false.