diff options
author | Tom Rondeau <trondeau@vt.edu> | 2012-12-18 15:36:17 -0500 |
---|---|---|
committer | Tom Rondeau <trondeau@vt.edu> | 2012-12-18 15:36:17 -0500 |
commit | 34c9ee546b0fba7b3938170bf228e5f6c2991fdb (patch) | |
tree | 658dd468e76a89988b548d9af575bd8227ca59b4 /docs/doxygen | |
parent | e39fae92a396aa984cba96a8bfd5cfe562ed7008 (diff) | |
parent | 620dd7ece3789220d0b46259a95752d8da7af730 (diff) |
Merge branch 'master' into next
Diffstat (limited to 'docs/doxygen')
-rw-r--r-- | docs/doxygen/other/main_page.dox | 6 | ||||
-rw-r--r-- | docs/doxygen/other/msg_passing.dox | 269 | ||||
-rw-r--r-- | docs/doxygen/other/pmt.dox | 348 |
3 files changed, 623 insertions, 0 deletions
diff --git a/docs/doxygen/other/main_page.dox b/docs/doxygen/other/main_page.dox index 15a56d0dec..681dea41c8 100644 --- a/docs/doxygen/other/main_page.dox +++ b/docs/doxygen/other/main_page.dox @@ -37,6 +37,12 @@ More details on packages in GNU Radio: \li \ref page_qtgui \li \ref page_uhd \li \ref page_vocoder + +More details on GNU Radio concepts: +\li \ref page_pmt +\li \ref page_msg_passing +\li \ref page_metadata +\li \ref volk_guide \li \ref page_pfb diff --git a/docs/doxygen/other/msg_passing.dox b/docs/doxygen/other/msg_passing.dox new file mode 100644 index 0000000000..aea0ac94ae --- /dev/null +++ b/docs/doxygen/other/msg_passing.dox @@ -0,0 +1,269 @@ +/*! \page page_msg_passing Message Passing + +\section intro Introduction + +GNU Radio was originally a streaming system with no other mechanism to +pass data between blocks. Streams of data are a model that work well +for samples, bits, etc., but are not really the right mechanism for +control data, metadata, and, often, packet structures (at least at +some point in the processing chain). + +We solved part of this problem a few years ago by introducing the tag +stream. This is a parallel stream to the data streaming. The +difference is that tags are designed to hold metadata and control +information. Tags are specifically associated with a particular sample +in the data stream and flow downstream alongside the data. This model +allows other blocks to identify that an event or action has occurred +or should occur on a particular item. The major limitation is that the +tag stream is really only accessible inside a work function and only +flows in one direction. Its benefit is that it is isosynchronous with +the data. + +We want a more general message passing system for a couple of +reasons. The first is to allow blocks downstream to communicate back +to blocks upstream. The second is to allow an easier way for us to +communicate back and forth between external applications and GNU +Radio. The new message passing interface handles these cases, although +it does so on an asynchronous basis. + +The message passing interface heavily relies on Polymorphic Types +(PMTs) in GNU Radio. For further information about these data +structures, see the page \ref page_pmt. + +\section api Message Passing API + +The message passing interface is designed into the gr_basic_block, +which is the parent class for all blocks in GNU Radio. Each block has +a set of message queues to hold incoming messages and can post +messages to the message queues of other blocks. The blocks also +distinguish between input and output ports. + +A block has to declare its input and output message ports in its +constructor. The message ports are described by a name, which is in +practice a PMT symbol (<em>i.e.</em>, an interned string). The API calls +to register a new port are: + +\code + void message_port_register_in(pmt::pmt_t port_id) + void message_port_register_out(pmt::pmt_t port_id) +\endcode + +The ports are now identifiable by that port name. Other blocks who may +want to post or receive messages on a port must subscribe to it. When +a block has a message to send, they are published on a particular +port. The subscribe and publish API looks like: + +\code + void message_port_pub(pmt::pmt_t port_id, + pmt::pmt_t msg); + void message_port_sub(pmt::pmt_t port_id, + pmt::pmt_t target); + void message_port_unsub(pmt::pmt_t port_id, + pmt::pmt_t target); +\endcode + +Any block that has a subscription to another block's output message +port will receive the message when it is published. Internally, when a +block publishes a message, it simply iterates through all blocks that +have subscribed and uses the gr_basic_block::_post method to send the +message to that block's message queue. + +From the flowgraph level, we have instrumented a gr_hier_block2::msg_connect +method to make it easy to subscribe blocks to other blocks' +messages. The message connection method looks like the following +code. Assume that the block \b src has an output message port named +\a pdus and the block \b dbg has an input port named \a print. + +\code + self.tb.msg_connect(src, "pdus", dbg, "print") +\endcode + +All messages published by the \b src block on port \a pdus will be +received by \b dbg on port \a print. Note here how we are just using +strings to define the ports, not PMT symbols. This is a convenience to +the user to be able to more easily type in the port names (for +reference, you can create a PMT symbol in Python using the +pmt::pmt_intern function as pmt.pmt_intern("string")). + +Users can also query blocks for the names of their input and output +ports using the following API calls: + +\code + pmt::pmt_t message_ports_in(); + pmt::pmt_t message_ports_out(); +\endcode + +The return value for these are a PMT vector filled with PMT symbols, +so PMT operators must be used to manipulate them. + +Each block has internal methods to handle posting and receiving of +messages. The gr_basic_block::_post method takes in a message and +places it into its queue. The publishing model uses the +gr_basic_block::_post method of the blocks as the way to access the +message queue. So the message queue of the right name will have a new +message. Posting messages also has the benefit of waking up the +block's thread if it is in a wait state. So if idle, as soon as a +message is posted, it will wake up and and call the message handler. + +The other side of the action in a block is in the message +handler. When a block has an input message port, it needs a callback +function to handle messages received on that port. We use a Boost bind +operator to bind the message port to the message handling +function. When a new message is pushed onto a port's message queue, +it is this function that is used to process the message. + + +\section examples Code Examples + +The following is snippets of code from blocks current in GNU Radio +that take advantage of message passing. We will be using +gr_message_debug and gr_tagged_stream_to_pdu below to show setting up +both input and output message passing capabilities. + +The gr_message_debug block is used for debugging the message passing +system. It describes two input message ports: \a print and \a +store. The \a print port simply prints out all messages to standard +out while the \a store port keeps a list of all messages posted to +it. This latter port works in conjunction with a +gr_message_debug::get_message(int i) call that allows us to retrieve +message \p i afterward. + +The constructor of this block looks like this: + +\code +{ + message_port_register_in(pmt::mp("print")); + set_msg_handler(pmt::mp("print"), + boost::bind(&gr_message_debug::print, this, _1)); + + message_port_register_in(pmt::mp("store")); + set_msg_handler(pmt::mp("store"), + boost::bind(&gr_message_debug::store, this, _1)); +} +\endcode + +So the two ports are registered by their respective names. We then use +the gr_basic_block::set_msg_handler function to identify this +particular port name with a callback function. The Boost \a bind +function (<a target="_blank" +href="http://www.boost.org/doc/libs/1_52_0/libs/bind/bind.html">Boost::bind</a>) +here binds the callback to a function of this block's class. So now +the block's gr_message_debug::print and gr_message_debug::store +functions are assigned to handle messages passed to them. Below is the +\a print function for reference. + +\code +void +gr_message_debug::print(pmt::pmt_t msg) +{ + std::cout << "***** MESSAGE DEBUG PRINT ********\n"; + pmt::pmt_print(msg); + std::cout << "**********************************\n"; +} +\endcode + +The function simply takes in the PMT message and prints it. The method +pmt::pmt_print is a function in the PMT library to print the +PMT in a friendly, (mostly) pretty manner. + +The gr_tagged_stream_to_pdu block only defines a single +output message port. In this case, its constructor looks like: + +\code +{ + message_port_register_out(pdu_port_id); +} +\endcode + +So we are only creating a single output port where \a pdu_port_id +is defined in the file gr_pdu.h as \a pdus. + +This blocks purpose is to take in a stream of samples along with +stream tags and construct a predefined PDU message from this. In GNU +Radio, we define a PDU as a PMT pair of (metadata, data). The metadata +describes the samples found in the data portion of the +pair. Specifically, the metadata can contain the length of the data +segment and any other information (sample rate, etc.). The PMT vectors +know their own length, so the length value is not actually necessary +unless useful for purposes down the line. The metadata is a PMT +dictionary while the data segment is a PMT uniform vector of either +bytes, floats, or complex values. + +In the end, when a PDU message is ready, the block calls its +gr_tagged_stream_to_pdu::send_message function that is shown below. + +\code +void +gr_tagged_stream_to_pdu::send_meassage() +{ + if(pmt::pmt_length(d_pdu_vector) != d_pdu_length) { + throw std::runtime_error("msg length not correct"); + } + + pmt::pmt_t msg = pmt::pmt_cons(d_pdu_meta, + d_pdu_vector); + message_port_pub(pdu_port_id, msg); + + d_pdu_meta = pmt::PMT_NIL; + d_pdu_vector = pmt::PMT_NIL; + d_pdu_length = 0; + d_pdu_remain = 0; + d_inpdu = false; +} +\endcode + +This function does a bit of checking to make sure the PDU is ok as +well as some cleanup in the end. But it is the line where the message +is published that is important to this discussion. Here, the block +posts the PDU message to any subscribers by calling +gr_basic_block::message_port_pub publishing method. + +There is similarly a gr_pdu_to_tagged_stream block that essentially +does the opposite. It acts as a source to a flowgraph and waits for +PDU messages to be posted to it on its input port \a pdus. It extracts +the metadata and data and processes them. The metadata dictionary is +split up into key:value pairs and stream tags are created out of +them. The data is then converted into an output stream of items and +passed along. The next section describes how PDUs can be passed into a +flowgraph using the gr_pdu_to_tagged_stream block. + +\section posting Posting from External Sources + +The last feature of the message passing architecture to discuss here +is how it can be used to take in messages from an external source. We +can call a block's gr_basic_block::_post method directly and pass it a +message. So any block with an input message port can receive messages +from the outside in this way. + +The following example uses a gr_pdu_to_tagged_stream block +as the source block to a flowgraph. Its purpose is to wait for +messages as PDUs posted to it and convert them to a normal stream. The +payload will be sent on as a normal stream while the meta data will be +decoded into tags and sent on the tagged stream. + +So if we have created a \b src block as a PDU to stream, it has a \a +pdus input port, which is how we will inject PDU messages to the +flowgraph. These PDUs could come from another block or flowgraph, but +here, we will create and insert them by hand. + +\code + port = pmt.pmt_intern("pdus") + msg = pmt.pmt_cons(pmt.PMT_NIL, + pmt.pmt_make_u8vector(16, 0xFF)) + src.to_basic_block()._post(port, msg) +\endcode + +The PDU's metadata section is empty, hence the pmt::PMT_NIL +object. The payload is now just a simple vector of 16 bytes of all +1's. To post the message, we have to access the block's gr_basic_block +class, which we do using the gr_basic_block::to_basic_block method and +then call the gr_basic_block::_post method to pass the PDU to the +right port. + +All of these mechanisms are explored and tested in the QA code of the +file qa_pdu.py. + +There are some examples of using the message passing infrastructure +through GRC in gnuradio-core/src/examples/msg_passing. + +*/ diff --git a/docs/doxygen/other/pmt.dox b/docs/doxygen/other/pmt.dox new file mode 100644 index 0000000000..61b73bca13 --- /dev/null +++ b/docs/doxygen/other/pmt.dox @@ -0,0 +1,348 @@ +/*! \page page_pmt Polymorphic Types + +\section intro Introduction + +Polymorphic Types are opaque data types that are designed as generic +containers of data that can be safely passed around between blocks and +threads in GNU Radio. They are heavily used in the stream tags and +message passing interfaces. The most complete list of PMT function is, +of course, the source code, specifically the header file pmt.h. This +manual page summarizes the most important features and points of PMTs. + + +\section datatype PMT Data Type + +All PMTs are of the type pmt::pmt_t. This is an opaque container and +PMT functions must be used to manipulate and even do things like +compare PMTs. PMTs are also \a immutable (except PMT vectors). We +never change the data in a PMT; instead, we create a new PMT with the +new data. The main reason for this is thread safety. We can pass PMTs +as tags and messages between blocks and each receives its own copy +that we can read from. However, we can never write to this object, and +so if multiple blocks have a reference to the same PMT, there is no +possibility of thread-safety issues of one reading the PMT data while +another is writing the data. If a block is trying to write new data to +a PMT, it actually creates a new PMT to put the data into. Thus we +allow easy access to data in the PMT format without worrying about +mutex locking and unlocking while manipulating them. + +PMTs can represent the following: + +- Boolean values of true/false +- Strings (as symbols) +- Integers (long and uint64) +- Floats (as doubles) +- Complex (as two doubles) +- Pairs +- Tuples +- Vectors (of PMTs) +- Uniform vectors (of any standard data type) +- Dictionaries (list of key:value pairs) +- Any (contains a boost::any pointer to hold anything) + +The PMT library also defines a set of functions that operate directly +on PMTs such as: + +- Equal/equivalence between PMTs +- Length (of a tuple or vector) +- Map (apply a function to all elements in the PMT) +- Reverse +- Get a PMT at a position in a list +- Serialize and deserialize +- Printing + +The constants in the PMT library are: + +- pmt::PMT_T - a PMT True +- pmt::PMT_F - a PMT False +- pmt::PMT_NIL - an empty PMT (think Python's 'None') + +\section insert Inserting and Extracting Data + +Use pmt.h for a complete guide to the list of functions used to create +PMTs and get the data from a PMT. When using these functions, remember +that while PMTs are opaque and designed to hold any data, the data +underneath is still a C++ typed object, and so the right type of +set/get function must be used for the data type. + +Typically, a PMT object can be made from a scalar item using a call +like "pmt::pmt_from_<type>". Similarly, when getting data out of a +PMT, we use a call like "pmt::pmt_to_<type>". For example: + +\code +double a = 1.2345; +pmt::pmt_t pmt_a = pmt::pmt_from_double(a); +double b = pmt::pmt_to_double(pmt_a); + +int c = 12345; +pmt::pmt_t pmt_c = pmt::pmt_from_long(c); +int d = pmt::pmt_to_long(pmt_c); +\endcode + +As a side-note, making a PMT from a complex number is not obvious: + +\code +std::complex<double> a(1.2, 3.4); +pmt::pmt_t pmt_a = pmt::pmt_make_rectangular(a.real(), b.imag()); +std::complex<double> b = pmt::pmt_to_complex(pmt_a); +\endcode + +Pairs, dictionaries, and vectors have different constructors and ways +to manipulate them, and these are explained in their own sections. + + +\section strings Strings + +PMTs have a way of representing short strings. These strings are +actually stored as interned symbols in a hash table, so in other +words, only one PMT object for a given string exists. If creating a +new symbol from a string, if that string already exists in the hash +table, the constructor will return a reference to the existing PMT. + +We create strings with the following functions, where the second +function, pmt::pmt_intern, is simply an alias of the first. + +\code +pmt::pmt_t str0 = pmt::pmt_string_to_symbol(std::string("some string")); +pmt::pmt_t str1 = pmt::pmt_intern(std::string("some string")); +\endcode + +The string can be retrieved using the inverse function: + +\code +std::string s = pmt::pmt_symbol_to_string(str0); +\endcode + + +\section tests Tests and Comparisons + +The PMT library comes with a number of functions to test and compare +PMT objects. In general, for any PMT data type, there is an equivalent +"pmt::pmt_is_<type>". We can use these to test the PMT before trying +to access the data inside. Expanding our examples above, we have: + +\code +pmt::pmt_t str0 = pmt::pmt_string_to_symbol(std::string("some string")); +if(pmt::pmt_is_symbol(str0)) + std::string s = pmt::pmt_symbol_to_string(str0); + +double a = 1.2345; +pmt::pmt_t pmt_a = pmt::pmt_from_double(a); +if(pmt::pmt_is_double(pmt_a)) + double b = pmt::pmt_to_double(pmt_a); + +int c = 12345; +pmt::pmt_t pmt_c = pmt::pmt_from_long(c); +if(pmt::pmt_is_long(pmt_a)) + int d = pmt::pmt_to_long(pmt_c); + +\\ This will fail the test. Otherwise, trying to coerce \b pmt_c as a +\\ double when internally it is a long will result in an exception. +if(pmt::pmt_is_double(pmt_a)) + double d = pmt::pmt_to_double(pmt_c); + +\endcode + + +\section dict Dictionaries + +PMT dictionaries and lists of key:value pairs. They have a +well-defined interface for creating, adding, removing, and accessing +items in the dictionary. Note that every operation that changes the +dictionary both takes a PMT dictionary as an argument and returns a +PMT dictionary. The dictionary used as an input is not changed and the +returned dictionary is a new PMT with the changes made there. + +The following is a list of PMT dictionary functions. Click through to +get more information on what each does. + +- bool pmt::pmt_is_dict(const pmt_t &obj) +- pmt_t pmt::pmt_make_dict() +- pmt_t pmt::pmt_dict_add(const pmt_t &dict, const pmt_t &key, const pmt_t &value) +- pmt_t pmt::pmt_dict_delete(const pmt_t &dict, const pmt_t &key) +- bool pmt::pmt_dict_has_key(const pmt_t &dict, const pmt_t &key) +- pmt_t pmt::pmt_dict_ref(const pmt_t &dict, const pmt_t &key, const pmt_t ¬_found) +- pmt_t pmt::pmt_dict_items(pmt_t dict) +- pmt_t pmt::pmt_dict_keys(pmt_t dict) +- pmt_t pmt::pmt_dict_values(pmt_t dict) + +This example does some basic manipulations of PMT dictionaries in +Python. Notice that we pass the dictionary \a a and return the results +to \a a. This still creates a new dictionary and removes the local +reference to the old dictionary. This just keeps our number of +variables small. + +\code +from gruel import pmt + +key0 = pmt.pmt_intern("int") +val0 = pmt.pmt_from_long(123) +val1 = pmt.pmt_from_long(234) + +key1 = pmt.pmt_intern("double") +val2 = pmt.pmt_from_double(5.4321) + +# Make an empty dictionary +a = pmt.pmt_make_dict() + +# Add a key:value pair to the dictionary +a = pmt.pmt_dict_add(a, key0, val0) +pmt.pmt_print(a) + +# Add a new value to the same key; +# new dict will still have one item with new value +a = pmt.pmt_dict_add(a, key0, val1) +pmt.pmt_print(a) + +# Add a new key:value pair +a = pmt.pmt_dict_add(a, key1, val2) +pmt.pmt_print(a) + +# Test if we have a key, then delete it +print pmt.pmt_dict_has_key(a, key1) +a = pmt.pmt_dict_delete(a, key1) +print pmt.pmt_dict_has_key(a, key1) + +ref = pmt.pmt_dict_ref(a, key0, pmt.PMT_NIL) +pmt.pmt_print(ref) + +# The following should never print +if(pmt.pmt_dict_has_key(a, key0) and pmt.pmt_eq(ref, pmt.PMT_NIL)): + print "Trouble! We have key0, but it returned PMT_NIL" +\endcode + +\section vectors Vectors + +PMT vectors come in two forms: vectors of PMTs and vectors of uniform +data. The standard PMT vector is a vector of PMTs, and each PMT can be +of any internal type. On the other hand, uniform PMTs are of a +specific data type which come in the form: + +- (u)int8 +- (u)int16 +- (u)int32 +- (u)int64 +- float32 +- float64 +- complex 32 (std::complex<float>) +- complex 64 (std::complex<double>) + +That is, the standard sizes of integers, floats, and complex types of +both signed and unsigned. + +Vectors have a well-defined interface that allows us to make, set, +get, and fill them. We can also get the length of a vector with +pmt::pmt_length. + +For standard vectors, these functions look like: + +- bool pmt::pmt_is_vector(pmt_t x) +- pmt_t pmt::pmt_make_vector(size_t k, pmt_t fill) +- pmt_t pmt::pmt_vector_ref(pmt_t vector, size_t k) +- void pmt::pmt_vector_set(pmt_t vector, size_t k, pmt_t obj) +- void pmt::pmt_vector_fill(pmt_t vector, pmt_t fill) + +Uniform vectors have the same types of functions, but they are data +type-dependent. The following list tries to explain them where you +substitute the specific data type prefix for \a dtype (prefixes being: +u8, u16, u32, u64, s8, s16, s32, s64, f32, f64, c32, c64). + +- bool pmt::pmt_is_(dtype)vector(pmt_t x) +- pmt_t pmt::pmt_make_(dtype)vector(size_t k, (dtype) fill) +- pmt_t pmt::pmt_init_(dtype)vector(size_t k, const (dtype*) data) +- pmt_t pmt::pmt_init_(dtype)vector(size_t k, const std::vector<dtype> data) +- pmt_t pmt::pmt_(dtype)vector_ref(pmt_t vector, size_t k) +- void pmt::pmt_(dtype)vector_set(pmt_t vector, size_t k, (dtype) x) +- const dtype* pmt::pmt_(dtype)vector_elements(pmt_t vector, size_t &len) +- dtype* pmt::pmt_(dtype)vector_writable_elements(pmt_t vector, size_t &len) + +\b Note: We break the contract with vectors. The 'set' functions +actually change the data underneath. It is important to keep track of +the implications of setting a new value as well as accessing the +'vector_writable_elements' data. Since these are mostly standard data +types, sets and gets are atomic, so it is unlikely to cause a great +deal of harm. But it's only unlikely, not impossible. Best to use +mutexes whenever manipulating data in a vector. + + +\subsection blob BLOB + +A BLOB is a 'binary large object' type. In PMT's, this is actually +just a thin wrapper around a u8vector. + +\section pairs Pairs + +Pairs are inspired by LISP 'cons' data types, so you will find the +language here comes from LISP. A pair is just a pair of PMT +objects. They are manipulated using the following functions: + +- bool pmt::pmt_is_pair (const pmt_t &obj): Return true if obj is a pair, else false +- pmt_t pmt::pmt_cons(const pmt_t &x, const pmt_t &y): construct new pair +- pmt_t pmt::pmt_car(const pmt_t &pair): get the car of the pair (first object) +- pmt_t pmt::pmt_cdr(const pmt_t &pair): get the cdr of the pair (second object) +- void pmt::pmt_set_car(pmt_t pair, pmt_t value): Stores value in the car field +- void pmt::pmt_set_cdr(pmt_t pair, pmt_t value): Stores value in the cdr field + + +\section serdes Serializing and Deserializing + +It is often important to hide the fact that we are working with PMTs +to make them easier to transmit, store, write to file, etc. The PMT +library has methods to serialize data into a string buffer or a +string and then methods to deserialize the string buffer or string +back into a PMT. We use this extensively in the metadata files (see +\ref page_metadata). + +- bool pmt::pmt_serialize(pmt_t obj, std::streambuf &sink) +- std::string pmt::pmt_serialize_str(pmt_t obj) +- pmt_t pmt::pmt_deserialize(std::streambuf &source) +- pmt_t pmt::pmt_deserialize_str(std::string str) + +For example, we will serialize the data above to make it into a string +ready to be written to a file and then deserialize it back to its +original PMT. + +\code +from gruel import pmt + +key0 = pmt.pmt_intern("int") +val0 = pmt.pmt_from_long(123) + +key1 = pmt.pmt_intern("double") +val1 = pmt.pmt_from_double(5.4321) + +# Make an empty dictionary +a = pmt.pmt_make_dict() + +# Add a key:value pair to the dictionary +a = pmt.pmt_dict_add(a, key0, val0) +a = pmt.pmt_dict_add(a, key1, val1) + +pmt.pmt_print(a) + +ser_str = pmt.pmt_serialize_str(a) +print ser_str + +b = pmt.pmt_deserialize_str(ser_str) +pmt.pmt_print(b) + +\endcode + +The line where we 'print ser_str' will print and parts will be +readable, but the point of serializing is not to make a human-readable +string. This is only done here as a test. + + +\section printing Printing + +We have used the pmt::pmt_print function in these examples to nicely +print the contents of a PMT. Another way to print the contents is +using the overloaded "<<" operator with a stream buffer object. In +C++, we can inline print the contents of a PMT like: + +\code +pmt::pmt_t a pmt::pmt_from_double(1.0); +std::cout << "The PMT a contains " << a << std::endl; +\endcode + +*/ |