diff options
Diffstat (limited to 'docs/doxygen/other')
| -rw-r--r-- | docs/doxygen/other/msg_passing.dox | 236 | ||||
| -rw-r--r-- | docs/doxygen/other/stream_tags.dox | 96 |
2 files changed, 173 insertions, 159 deletions
diff --git a/docs/doxygen/other/msg_passing.dox b/docs/doxygen/other/msg_passing.dox index 827ad4ee62..14de7bae4c 100644 --- a/docs/doxygen/other/msg_passing.dox +++ b/docs/doxygen/other/msg_passing.dox @@ -14,7 +14,7 @@ GNU Radio was originally a streaming system with no other mechanism to pass data between blocks. Streams of data are a model that work well for samples, bits, etc., but are not really the right mechanism for -control data, metadata, and, often, packet structures (at least at +control data, metadata, or packet structures (at least at some point in the processing chain). We solved part of this problem by introducing the tag stream (see \ref @@ -32,7 +32,7 @@ We want a more general message passing system for a couple of reasons. The first is to allow blocks downstream to communicate back to blocks upstream. The second is to allow an easier way for us to communicate back and forth between external applications and GNU -Radio. The new message passing interface handles these cases, although +Radio. GNU Radio's message passing interface handles these cases, although it does so on an asynchronous basis. The message passing interface heavily relies on Polymorphic Types @@ -57,20 +57,33 @@ to register a new port are: void message_port_register_out(pmt::pmt_t port_id) \endcode +In Python: + +\code + self.message_port_register_in(pmt.intern("port name")) + self.message_port_register_out(pmt.intern("port name")) +\endcode + The ports are now identifiable by that port name. Other blocks who may want to post or receive messages on a port must subscribe to it. When a block has a message to send, they are published on a particular -port. The subscribe and publish API looks like: +port using the following API: \code - void message_port_pub(pmt::pmt_t port_id, - pmt::pmt_t msg); - void message_port_sub(pmt::pmt_t port_id, - pmt::pmt_t target); - void message_port_unsub(pmt::pmt_t port_id, - pmt::pmt_t target); + void message_port_pub(pmt::pmt_t port_id, pmt::pmt_t msg); \endcode +In Python: + +\code + self.message_port_pub(pmt.intern("port name"), <pmt message>) +\endcode + +Subscribing is usually done in the form of connecting message ports +as part of the flowgraph, as discussed later. Internally, when message +ports are connected, the gr::basic_block::message_port_sub method is +called. + Any block that has a subscription to another block's output message port will receive the message when it is published. Internally, when a block publishes a message, it simply iterates through all blocks that @@ -91,6 +104,14 @@ Boost's 'bind' function: boost::bind(&block_class::message_handler_function, this, _1)); \endcode +In Python: + +\code + self.set_msg_handler(pmt.intern("port name"), <msg handler function>) +\endcode + +When a new message is pushed onto a port's message queue, +it is this function that is used to process the message. The 'port_id' is the same PMT as used when registering the input port. The 'block_class::message_handler_function' is the member function of the class designated to handle messages to this port. The @@ -104,16 +125,22 @@ is: void block_class::message_handler_function(pmt::pmt_t msg); \endcode -We give an example of using this below. +In Python the equivalent function would be: + +\code + def handle_msg(self, msg): +\endcode + +We give examples of using this below. \subsection msg_passing_fg_connect Connecting Messages through the Flowgraph From the flowgraph level, we have instrumented a gr::hier_block2::msg_connect method to make it easy to subscribe blocks to other blocks' -messages. The message connection method looks like the following -code. Assume that the block \b src has an output message port named -\a pdus and the block \b dbg has an input port named \a print. +messages. Assume that the block \b src has an output message port named +\a pdus and the block \b dbg has an input port named \a print. The message +connection in the flowgraph (in Python) looks like the following: \code self.tb.msg_connect(src, "pdus", dbg, "print") @@ -144,24 +171,92 @@ gr::basic_block::_post method of the blocks as the way to access the message queue. So the message queue of the right name will have a new message. Posting messages also has the benefit of waking up the block's thread if it is in a wait state. So if idle, as soon as a -message is posted, it will wake up and and call the message handler. +message is posted, it will wake up and call the message handler. -The other side of the action in a block is in the message -handler. When a block has an input message port, it needs a callback -function to handle messages received on that port. We use a Boost bind -operator to bind the message port to the message handling -function. When a new message is pushed onto a port's message queue, -it is this function that is used to process the message. +\section msg_passing_posting Posting from External Sources -\section msg_passing_python Message Passing in Python Blocks +An important feature of the message passing architecture +is how it can be used to take in messages from an external source. We +can call a block's gr::basic_block::_post method directly and pass it a +message. So any block with an input message port can receive messages +from the outside in this way. + +The following example uses a gr::blocks::pdu_to_tagged_stream block +as the source block to a flowgraph. Its purpose is to wait for +messages as PDUs posted to it and convert them to a normal stream. The +payload will be sent on as a normal stream while the meta data will be +decoded into tags and sent on the tagged stream. + +So if we have created a \b src block as a PDU to stream, it has a \a +pdus input port, which is how we will inject PDU messages into the +flowgraph. These PDUs could come from another block or flowgraph, but +here, we will create and insert them by hand. + +\code + port = pmt.intern("pdus") + msg = pmt.cons(pmt.PMT_NIL, pmt.make_u8vector(16, 0xFF)) + src.to_basic_block()._post(port, msg) +\endcode + +The PDU's metadata section is empty, hence the pmt::PMT_NIL +object. The payload is now just a simple vector of 16 bytes of all +1's. To post the message, we have to access the block's gr::basic_block +class, which we do using the gr::basic_block::to_basic_block method and +then call the gr::basic_block::_post method to pass the PDU to the +right port. + +All of these mechanisms are explored and tested in the QA code of the +file qa_pdu.py. + +There are some examples of using the message passing infrastructure +through GRC in gr-blocks/examples/msg_passing. + + +\section msg_passing_commands Using messages as commands + +One important use of messages is to send commands to blocks. Examples for this include: + +- gr::qtgui::freq_sink_c: The scaling of the frequency axis can be changed by messages +- gr::uhd::usrp_source and gr::uhd::usrp_sink: Many transceiver-related settings can + be manipulated through command messages, such as frequency, gain and LO offset +- gr::digital::header_payload_demux, which receives an acknowledgement from a header parser + block on how many payload items there are to process + +There is no special PMT type to encode commands, however, it is strongly recommended +to use one of the following formats: + +- pmt::cons(KEY, VALUE): This format is useful for commands that take a single value. + Think of KEY and VALUE as the argument name and value, respectively. For the case of + the QT GUI Frequency Sink, KEY would be "freq" and VALUE would be the new center frequency + in Hz. +- pmt::dict((KEY1: VALUE1), (KEY2: VALUE2), ...): This is basically the same as the + previous format, but you can provide multiple key/value pairs. This is particularly + useful when a single command takes multiple arguments which can't be broken into + multiple command messages (e.g., the USRP blocks might have both a timestamp and a + center frequency in a command message, which are closely associated). + +In both cases, all KEYs should be pmt::symbols (i.e. strings). VALUEs can be +whatever the block requires. + +It might be tempting to deviate from this format, e.g. the QT Frequency sink could +simply take a float value as a command message, and it would still work fine. +However, there are some very good reasons to stick to this format: -ADD STUFF HERE +- Interoperability: The more people use the standard format, the more likely it + is that blocks from different sources can work together +- Inspectability: A message debug block will display more useful information about + a message if it's containing both a value and a key +- Intuition: This format is pretty versatile and unlikely to create situations + where it is not sufficient (especially considering that values are PMTs themselves). + As a counterexample, using positional arguments (something like "the first argument + is the frequency, the second the gain") is easily forgotten, or changed in one place + and not another, etc. \section msg_passing_examples Code Examples -The following is snippets of code from blocks current in GNU Radio +The following is snippets of code from blocks currently in GNU Radio that take advantage of message passing. We will be using gr::blocks::message_debug and gr::blocks::tagged_stream_to_pdu below to show setting up both input and output message passing capabilities. @@ -193,10 +288,10 @@ The constructor of this block looks like this: } \endcode -So the three ports are registered by their respective names. We then -use the gr::basic_block::set_msg_handler function to identify this -particular port name with a callback function. The Boost \a bind -function (<a target="_blank" +The three message input ports are registered by their respective +names. We then use the gr::basic_block::set_msg_handler function to +identify this particular port name with a callback function. The +Boost \a bind function (<a target="_blank" href="http://www.boost.org/doc/libs/1_52_0/libs/bind/bind.html">Boost::bind</a>) here binds the callback to a function of this block's class. So now the functions in the block's private implementation class, @@ -217,7 +312,7 @@ message_debug_impl::print(pmt::pmt_t msg) The function simply takes in the PMT message and prints it. The method pmt::print is a function in the PMT library to print the -PMT in a friendly, (mostly) pretty manner. +PMT in a friendly and (mostly) pretty manner. The gr::blocks::tagged_stream_to_pdu block only defines a single output message port. In this case, its constructor contains the line: @@ -231,8 +326,8 @@ output message port. In this case, its constructor contains the line: So we are only creating a single output port where \a pdu_port_id is defined in the file pdu.h as \a pdus. -This blocks purpose is to take in a stream of samples along with -stream tags and construct a predefined PDU message from this. In GNU +This block's purpose is to take in a stream of samples along with +stream tags and construct a predefined PDU message from it. In GNU Radio, we define a PDU as a PMT pair of (metadata, data). The metadata describes the samples found in the data portion of the pair. Specifically, the metadata can contain the length of the data @@ -266,7 +361,7 @@ tagged_stream_to_pdu_impl::send_message() } \endcode -This function does a bit of checking to make sure the PDU is ok as +This function does a bit of checking to make sure the PDU is OK as well as some cleanup in the end. But it is the line where the message is published that is important to this discussion. Here, the block posts the PDU message to any subscribers by calling @@ -281,83 +376,6 @@ them. The data is then converted into an output stream of items and passed along. The next section describes how PDUs can be passed into a flowgraph using the gr::blocks::pdu_to_tagged_stream block. -\section msg_passing_posting Posting from External Sources - -The last feature of the message passing architecture to discuss here -is how it can be used to take in messages from an external source. We -can call a block's gr::basic_block::_post method directly and pass it a -message. So any block with an input message port can receive messages -from the outside in this way. - -The following example uses a gr::blocks::pdu_to_tagged_stream block -as the source block to a flowgraph. Its purpose is to wait for -messages as PDUs posted to it and convert them to a normal stream. The -payload will be sent on as a normal stream while the meta data will be -decoded into tags and sent on the tagged stream. - -So if we have created a \b src block as a PDU to stream, it has a \a -pdus input port, which is how we will inject PDU messages to the -flowgraph. These PDUs could come from another block or flowgraph, but -here, we will create and insert them by hand. - -\code - port = pmt.intern("pdus") - msg = pmt.cons(pmt.PMT_NIL, - pmt.make_u8vector(16, 0xFF)) - src.to_basic_block()._post(port, msg) -\endcode - -The PDU's metadata section is empty, hence the pmt::PMT_NIL -object. The payload is now just a simple vector of 16 bytes of all -1's. To post the message, we have to access the block's gr::basic_block -class, which we do using the gr::basic_block::to_basic_block method and -then call the gr::basic_block::_post method to pass the PDU to the -right port. - -All of these mechanisms are explored and tested in the QA code of the -file qa_pdu.py. - -There are some examples of using the message passing infrastructure -through GRC in gr-blocks/examples/msg_passing. - -\section msg_passing_commands Using messages as commands - -Messages can be used to send commands to blocks. Examples for this include: - -- gr::qtgui::freq_sink_c: The scaling of the frequency axis can be changed by messages -- gr::uhd::usrp_source and gr::uhd::usrp_sink: Many transceiver-related settings can - be manipulated through command messages, such as frequency, gain and LO offset -- gr::digital::header_payload_demux, which receives an acknowledgement from a header parser - block on how many payload items there are to process - -There is no special PMT type to encode commands, however, it is strongly recommended -to use one of the following formats: - -- pmt::cons(KEY, VALUE): This format is useful for commands that take a single value. - Think of KEY and VALUE as the argument name and value, respectively. For the case of - the QT GUI Frequency Sink, KEY would be "freq" and VALUE would be the new center frequency - in Hz. -- pmt::dict((KEY1: VALUE1), (KEY2: VALUE2), ...): This is basically the same as the - previous format, but you can provide multiple key/value pairs. This is particularly - useful when a single command takes multiple arguments which can't be broken into - multiple command messages (e.g., the USRP blocks might have both a timestamp and a - center frequency in a command message, which are closely associated). - -In both cases, all KEYs should be pmt::symbols (i.e. strings). VALUEs can be -whatever the block requires. - -It might be tempting to deviate from this format, e.g. the QT Frequency sink could -simply take a float value as a command message, and it would still work fine. -However, there are some very good reasons to stick to this format: - -- Interoperability: The more people use the standard format, the more likely it - is that blocks from different sources can work together -- Inspectability: A message debug block will display more useful information about - a message if its containing both a value and a key -- Intuition: This format is pretty versatile and unlikely to create situations - where it is not sufficient (especially considering that values are PMTs themselves). - As a counterexample, using positional arguments (something like "the first argument - is the frequency, the second the gain") is easily forgotten, or changed in one place - and not another, etc. +For a Python block example, see \ref pyblocks_msgs. */ diff --git a/docs/doxygen/other/stream_tags.dox b/docs/doxygen/other/stream_tags.dox index 8edc598e96..146218796e 100644 --- a/docs/doxygen/other/stream_tags.dox +++ b/docs/doxygen/other/stream_tags.dox @@ -15,7 +15,7 @@ GNU Radio was originally a streaming system with no other mechanism to pass data between blocks. Streams of data are a model that work well for samples, bits, etc., but can lack for control and meta data. -Part of this is solved using the message passing interface, which +Part of this is solved using the existing message passing interface, which allows blocks to subscribe to messages published by any other block in the flowgraph (see \ref page_msg_passing). The main drawback to the message passing system is that is works asynchronously, meaning that @@ -24,18 +24,18 @@ stream. Stream tags are an isosynchronous data stream that runs parallel to the main data stream. A stream \a tag is generated by a block's work -function and from there on flows downstream with a particular sample +function and from there on flows downstream alongside a particular sample, until it reaches a sink or is forced to stop propagating by another block. Stream tags are defined for a specific item in the data stream and are -formed as a key:value pair. The \a key identifies what the \a value is +formed as a key:value pair. The \a key identifies what the \a value represents while the value holds the data that the tag contains. Both \a key and \a value are PMTs (\ref page_pmt) where the \a key is a PMT symbol while -the \a value any type of PMT and can therefore handle any data we wish -to pass. A fourth part of the tag is the \a srcid, which is a PMT +the \a value is any type of PMT and can therefore handle any data we wish +to pass. An additional part of the tag is the \a srcid, which is a PMT symbol and is used to identify the block that created the tag (which -is usually the block's alias()). +is usually the block's alias). \section stream_tags_block_api_extensions API Extensions to the gr::block @@ -45,10 +45,10 @@ understand \a absolute item numbers. In the data stream model, each block's work function is given a buffer in the data stream that is referenced from 0 to N-1. This is a \a relative offset into the data stream. The absolute reference starts from the beginning of the -flowgraph and continues to count up with ever item. Each input stream +flowgraph and continues to count up with every item. Each input stream is associated with a concept of the 'number of items read' and each -output stream has a 'number of items written.' These are programmed -using the two API calls: +output stream has a 'number of items written'. These are retrieved during +runtime using the two API calls: \code unsigned long int nitems_read(unsigned int which_input); @@ -69,40 +69,25 @@ at <em>nitems_written(0)+i</em> for the 0th output port. \section stream_tags_api Stream Tags API -The stream tags API consists of four functions, two to add and two to -get the stream tags. These functions are only meant to be accessed -within a call to general_work/work. While they can be called elsewhere +The stream tags API is split into two parts: adding tags to a stream, +and getting tags from a stream. +Note that the functions described below are only meant to be accessed +within a call to general_work/work. While they can be called at other points in time by a block, the behavior outside of work is undefined without exact knowledge of the item counts in the buffers. -\li gr::block::add_item_tag: Adds an item tag to a particular output port using a -gr::tag_t data type. -\li gr::block::add_item_tag: Adds an item tag to a particular output -port where each value of the tag is explicitly given. - -\li gr::block::get_tags_in_range: Gets all tags from a particular -input port between a certain range of items (in absolute item time). - -\li gr::block::get_tags_in_range: Gets any tag that has a specified -key from a particular input port between a certain range of items (in -absolute item time). - -\li gr::block::get_tags_in_window: Gets all tags from a particular -input port between a certain range of items (in relative item time -within the work function). - -\li gr::block::get_tags_in_range: Gets any tag that has a specified -key from a particular input port between a certain range of items (in -relative item time within the work function). +\subsection stream_tags_add_item_tag Adding a Tag to a Stream +We add a tag to a particular output stream of the block using: -\subsection stream_tags_add_item_tag Adding a Tag to a Stream +\li gr::block::add_item_tag: Adds an item tag to a particular output port +using a gr::tag_t data type or by specifying the tag values. -The two function calls to add items tags are defined here. We add a -tag to a particular output stream of the block. We can output them to -multiple output streams if we want, but to do so means calling one of -these functions once for each port. +We can output them to multiple output streams if we want, but to do so +means calling this function once for each port. This function can be +provided with a gr::tag_t data type, or each value of the tag can be +explicitly given. Again, a tag is defined as: @@ -136,14 +121,25 @@ of the tag information in the function call: \subsection stream_tags_get_item_tags Getting tags from a Stream -To get tags from a particular input stream, we again have two -functions we can use. Both of these pass back vectors of -gr::tag_t. The second function allows us to specify a particular key -(as a PMT symbol) that filters out all but the key we are interested -in, which reduces the effort inside the work function for getting the -right tag's data. +To get tags from a particular input stream, we have two +functions we can use: + +\li gr::block::get_tags_in_range: Gets all tags from a particular +input port between a certain range of items (in absolute item time). + +\li gr::block::get_tags_in_window: Gets all tags from a particular +input port between a certain range of items (in relative item time +within the work function). -The first call just returns any tags between the given range of items: +The difference between these functions is working in absolute item +time versus relative item time. Both of these pass back vectors of +gr::tag_t, and they both allow +specifying a particular key (as a PMT symbol) to filter against +(or the fifth argument can be left out to search for all keys). +Filtering for a certain key reduces the effort inside the work function +for getting the right tag's data. + +For example, this call just returns any tags between the given range of items: \code void get_tags_in_range(std::vector<tag_t> &v, @@ -167,7 +163,7 @@ key \a key. \section stream_tags_propagation Tag Propagation Tags are propagated downstream from block to block like the normal -data streams. How blocks are actually moved depends on a specific +data streams. How tags are actually moved depends on a specific propagation policy. We defined three types of policies: \li All-to-All: all tags from any input port are replicated to all @@ -175,7 +171,7 @@ output ports \li One-to-One: tags from input port \a i are only copied to output port \a i (depends on num inputs = num outputs). \li Dont: Does not propagate tags. Tags are either stopped here or the -work function propagates them itself. +work function recreates them in some manner. The default behavior of a block is the 'All-to-All' method of propagation. @@ -217,9 +213,9 @@ block. This becomes relevant when using \ref page_tagged_stream_blocks. Tags can be very useful to an application, and their use is spreading. USRP sources generate tag information on the time, sample rate, and frequency of the board if anything changes. We have a meta -data file source/sink that use tags to store information about the -data stream. But there are things to think about when using tags in a -block. +data file source/sink (see \ref page_metadata) that use tags to store +information about the data stream. But there are things to think about +when using tags in a block. First, when tags are not being used, there is almost no effect on the scheduler. However, when we use tags, we add overhead by getting and @@ -233,7 +229,7 @@ necessary and try to provide some control over how tags are generated to control their frequency. A good example is the USRP source, which generates a time tag. If it generated a tag with every sample, we would have thousands of tags per second, which would add a significant -amount of overhead. Conversely, if we started at time <em>t0</em> at +amount of overhead. This is because if we started at time <em>t0</em> at sample rate <em>sr</em>, then after <em>N</em> samples, we know that we are now at time <em>t0 + N/sr</em>. So continuously producing new tags adds no information. @@ -243,7 +239,7 @@ there is a discontinuity in the packets received from the USRP. Since we have no way of knowing in the flowgraph how many samples were potentially lost, we have lost track of the timing information. The USRP driver recognizes when packets have been dropped and uses this to -queue another tag, which allows us to resync. Likewise, any time the +queue another tag, which allows us to resync. Likewise, any point the sample rate or frequency changes, a new tag is issued. */ |