diff options
32 files changed, 4692 insertions, 36 deletions
diff --git a/docs/doxygen/other/metadata.dox b/docs/doxygen/other/metadata.dox new file mode 100644 index 0000000000..9fad7c584c --- /dev/null +++ b/docs/doxygen/other/metadata.dox @@ -0,0 +1,329 @@ +/*! \page page_metadata Metadata Information + +\section Introduction + +Metadata files have extra information in the form of headers that +carry metadata about the samples in the file. Raw, binary files carry +no extra information and must be handled delicately. Any changes in +the system state such as sample rate or if a receiver's frequency are +not conveyed with the data in the file itself. Header of metadata +solve this problem. + +We write metadata files using blocks::file_meta_sink and read metadata +files using blocks::file_meta_source. + +Metadata files have headers that carry information about a segment of +data within the file. The header structure is described in detail in +the next section. A metadata file always starts with a header that +describes the basic structure of the data. It contains information +about the item size, data type, if it's complex, the sample rate of +the segment, the time stamp of the first sample of the segment, and +information regarding the header size and segment size. + +Headers have two main tags associated with them: + +- rx_rate: the sample rate of the stream. +- rx_time: the time stamp of the first item in the segment. + +These tags were inspired by the UHD tag format. + +The header gives enough information to process and handle the +data. One cautionary note, though, is that the data type should never +change within a file. There should be very little need for this, but +more importantly. GNU Radio blocks can only set the data type of their +IO signatures in the constructor, so changes in the data type +afterward will not be recognized. + +We also have an extra header segment that is option. This can be +loaded up at the beginning by the user specifying some extra metadata +that should be transmitted along with the data. It also grows whenever +it sees a stream tag, so the dictionary will contain and key:value +pairs out of tags from the flowgraph. + + +\subsection types Types of Metadata Files + +GNU Radio currently supports two types of metadata files: + +- inline: headers are inline with the data in the same file. +- detached: headers are in a separate header file from the data. + +The inline method is the standard version. When a detached header is +used, the headers are simply inserted back-to-back in the detached +header file. The dat file, then, is the standard raw binary format +with no interruptions in the data. + + +\subsection updating Updating Headers + +While there is always a header that starts a metadata file, they are +updated throughout as well. There are two events that trigger a new +header. We define a segment as the unit of data associated with the +last header. + +The first event that will trigger a new header is when enough samples +have been written for the given segment. This number is defined as the +maximum segment size and is a parameter we pass to the +file_meta_sink. It defaults to 1 million items (items, not +bytes). When that number of items is reached, a new header is +generated and a new segment is started. This makes it easier for us to +manipulate the data later and helps protect against catastrophic data +loss. + +The second event to trigger a new segment is if a new tag is +observed. If the tag is a standard tag in the header, the header value +is updated, the header and current extras are written to file, and the +segment begins again. If a tag from the extras is seen, the value +associated with that tag is updated; and if a new tag is seen, a new +key:value pair are added to the extras dictionary. + +When new tags are seen, we generate a new segment so that we make sure +that all samples in that segment are defined by the header. If the +sample rate changes, we create a new segment where all of the new +samples are at that new rate. Also, in the case of UHD devices, if a +segment loss is observed, it will generate a new timestamp as a tag of +'rx_time'. We create a new file segment that reflects this change to +keep the sample times exact. + + +\subsection implementation Implementation + +Metadata files are created using file_meta_sink. The default +behavior is to create a single file with inline headers as +metadata. An option can be set to switch to detached header mode. + +Metadata file are read into a flowgraph using +file_meta_source. This source reads a metadata file, inline by +default with a settable option to use detached headers. The data from +the segments is converted into a standard streaming output. The +'rx_rate' and 'rx_time' and all key:value pairs in the extra header +are converted into tags and added to the stream tags interface. + + +\section structure Structure + +The file metadata consists of a static mandatory header and a dynamic +optional extras header. Each header is a separate PMT +dictionary. Headers are created by building a PMT dictionary of +key:value pairs, then the dictionary is serialized into a string to be +written to file. The header is always the same length that is +predetermined by the version of the header (this must be known +already). The header will then indicate if there is an extra data to +be extracted as a separate serialized dictionary. + +To work with the PMTs for creating and extracting header information, +we use PMT operators. For example, we create a simplified version of +the header in C++ like this: + +\code + using namespace pmt; + const char METADATA_VERSION = 0x0; + pmt_t header; + header = pmt_make_dict(); + header = pmt_dict_add(header, mp("version"), mp(METADATA_VERSION)); + header = pmt_dict_add(header, mp("rx_rate"), mp(samp_rate)); + std::string hdr_str = pmt_serialize_str(header); +\endcode + +The call to pmt_dict_add adds a new key:value pair to the +dictionary. Notice that it both takes and returns the 'header' +variable. This is because we are actually creating a new dictionary +with this function, so we just assign it to the same variable. + +The 'mp' functions are convenience functions provided by the PMT +library. They interpret the data type of the value being inserted and +call the correct 'pmt_from_xxx' function. For more direct control over +the data type, see PMT functions in pmt.h, such as pmt_from_uint64 or +pmt_from_double. + +We finish this off by using 'pmt_serialize_str' to convert the PMT +dictionary into a specialized string format that makes it easy to +write to a file. + +The header is always METADATA_HEADER_SIZE bytes long and a metadata +file always starts with a header. So to extract the header from a +file, we need to read in this many bytes from the beginning of the +file and deserialize it. An important note about this is that the +deserialize function must operate on a std::string. The serialized +format of a dictionary contains null characters, so normal C character +arrays (e.g., 'char *s') get confused. + +Assuming that 'std::string str' contains the full string as read from +a file, we can access the dictionary in C++ like this: + +\code + pmt_t hdr = pmt_deserialize_str(str); + if(pmt_dict_has_key(hdr, pmt_string_to_symbol("strt"))) { + pmt_t r = pmt_dict_ref(hdr, pmt_string_to_symbol("strt"), PMT_NIL); + uint64_t seg_start = pmt_to_uint64(r); + uint64_t extra_len = seg_start - METADATA_HEADER_SIZE; + } +\endcode + +This example first deserializes the string into a PMT dictionary +again. This will throw an error if the string is malformed and cannot +be deserialized correctly. We then want to get access to the item with +key 'strt'. As the next subsection will show, this value indicates at +which byte the data segment starts. We first check to make sure that +this key exists in the dictionary. If not, our header does not contain +the correct information and we might want to handle this as an error. + +Assuming the header is properly formatted, we then get the particular +item referenced by the key 'strt'. This is a uint64_t, so we use the +PMT function to extract and convert this value properly. We now know +if we have an extra header in the file by looking at the difference +between 'seg_start' and the static header size, +METADATA_HEADER_SIZE. If the 'extra_len' is greater than 0, we know we +have an extra header that we can process. Moreover, this also tells us +the size of the serialized PMT dictionary in bytes, so we can easily +read this many bytes from the file. We can then deserialize and parse +this header just like the first. + + +\subsection header Header Information + +The header is a PMT dictionary with a known structure. This structure +may change, but we version the headers, so all headers of version X +must be the same length and structure. As of now, we only have version +0 headers, which look like the following: + +- version: (char) version number (usually set to METADATA_VERSION) +- rx_rate: (double) Stream's sample rate +- rx_time: (pmt_t pair - (uint64_t, double)) Time stamp (format from UHD) +- size: (int) item size in bytes - reflects vector length if any. +- type: (int) data type (enum below) +- cplx: (bool) true if data is complex +- strt: (uint64_t) start of data relative to current header +- bytes: (uint64_t) size of following data segment in bytes + +The data types are indicated by an integer value from the following +enumeration type: + +\code +enum gr_file_types { + GR_FILE_BYTE=0, + GR_FILE_CHAR=0, + GR_FILE_SHORT=1, + GR_FILE_INT, + GR_FILE_LONG, + GR_FILE_LONG_LONG, + GR_FILE_FLOAT, + GR_FILE_DOUBLE, +}; +\endcode + +\subsection extras Extras Information + +The extras section is an optional segment of the header. If 'strt' == +METADATA_HEADER_SIZE, then there is no extras. Otherwise, it is simply +a PMT dictionary of key:value pairs. The extras header can contain +anything and can grow while a program is running. + +We can insert extra data into the header at the beginning if we +wish. All we need to do is use the 'pmt_dict_add' function to insert +our hand-made metadata. This can be useful to add our own markers and +information. + +The main role of the extras header, though, is as a container to hold +any stream tags. When a stream tag is observed coming in, the tag's +key and value are added to the dictionary. Like a standard dictionary, +any time a key already exists, the value will be updated. If the key +does not exist, a new entry is created and the new key:value pair are +added together. So any new tags that the file metadata sink sees will +add to the dictionary. It is therefore important to always check the +'strt' value of the header to see if the length of the extras +dictionary has changed at all. + +When reading out data from the extras, we do not necessarily know the +data type of the PMT value. The key is always a PMT symbol, but the +value can be any other PMT type. There are PMT functions that allow us +to query the PMT to test if it is a particular type. We also have the +ability to do 'pmt_print' on any PMT object to print it to +screen. Before converting from a PMT to it's natural data type, it is +necessary to know the data type. + + +\section Utilities + +GNU Radio comes with a couple of utilities to help in debugging and +manipulating metadata files. There is a general parser in Python that +will convert the PMT header and extra header into Python +dictionaries. This utility is: + +- gr-blocks/python/parse_file_metadata.py + +This program is installed into the Python directory under the +'gnuradio' module, so it can be accessed with: + +\code +from gnuradio.blocks import parse_file_metadata +\endcode + +It defines HEADER_LENGTH as the static length of the metadata header +size. It also has dictionaries that can be used to convert from the +file type to a string (ftype_to_string) and one to convert from the +file type to the size of the data type in bytes (ftype_to_size). + +The 'parse_header' takes in a PMT dictionary, parses it, and returns a +Python dictionary. An optional 'VERBOSE' bool can be set to print the +information to standard out. + +The 'parse_extra_dict' is similar in that it converts from a PMT +dictionary to a Python dictionary. The values are kept in their PMT +format since we do not necessarily know the native data type. + +A program called 'gr_read_file_metadata' is installed into the path +and can be used to read out all header information from a metadata +file. This program is just called with the file name as the first +command-line argument. An option '-D' will handle detached header +files where the file of headers is expected to be the file name of the +data with '.hdr' appended to it. + + +\section Examples + +Examples are located in: + +- gr-blocks/examples/metadata + +Currently, there are a few GRC example programs. + +- file_metadata_sink: create a metadata file from UHD samples. +- file_metadata_source: read the metadata file as input to a simple graph. +- file_metadata_vector_sink: create a metadata file from UHD samples. +- file_metadata_vector_source: read the metadata file as input to a simple graph. + +The file sink example can be switched to use a signal source instead +of a UHD source, but no extra tagged data is used in this mode. + +The file source example pushes the data stream to a new raw file while +a tag debugger block prints out any tags observed in the metedata +file. A QT GUI time sink is used to look at the signal as well. + +The versions with 'vector' in the name are similar except they use +vectors of data. + +The following shows a simple way of creating extra metadata for a +metadata file. This example is just showing how we can insert a date +into the metadata to keep track of later. The date in this case is +encoded as a vector of uint16 with [day, month, year]. + +\code + from gruel import pmt + from gnuradio import blocks + + key = pmt.pmt_intern("date") + val = pmt.pmt_init_u16vector(3, [13,12,2012]) + + extras = pmt.pmt_make_dict() + extras = pmt.pmt_dict_add(extras, key, val) + extras_str = pmt.pmt_serialize_str(extras) + self.sink = blocks.file_meta_sink(gr.sizeof_gr_complex, + "/tmp/metadat_file.out", + samp_rate, 1, + blocks.GR_FILE_FLOAT, True, + 1000000, extra_str, False) + +\endcode + +*/ diff --git a/gnuradio-core/src/lib/general/gr_tag_debug.cc b/gnuradio-core/src/lib/general/gr_tag_debug.cc index c4031f438d..5833a18912 100644 --- a/gnuradio-core/src/lib/general/gr_tag_debug.cc +++ b/gnuradio-core/src/lib/general/gr_tag_debug.cc @@ -65,10 +65,11 @@ gr_tag_debug::work(int noutput_items, { gruel::scoped_lock l(d_mutex); + std::stringstream sout; if(d_display) { - std::cout << std::endl - << "----------------------------------------------------------------------"; - std::cout << std::endl << "Tag Debug: " << d_name << std::endl; + sout << std::endl + << "----------------------------------------------------------------------"; + sout << std::endl << "Tag Debug: " << d_name << std::endl; } uint64_t abs_N, end_N; @@ -80,20 +81,23 @@ gr_tag_debug::work(int noutput_items, get_tags_in_range(d_tags, i, abs_N, end_N); if(d_display) { - std::cout << "Input Stream: " << i << std::endl; + sout << "Input Stream: " << i << std::endl; for(d_tags_itr = d_tags.begin(); d_tags_itr != d_tags.end(); d_tags_itr++) { - std::cout << std::setw(10) << "Offset: " << d_tags_itr->offset - << std::setw(10) << "Source: " << pmt::pmt_symbol_to_string(d_tags_itr->srcid) - << std::setw(10) << "Key: " << pmt::pmt_symbol_to_string(d_tags_itr->key) - << std::setw(10) << "Value: "; - pmt::pmt_print(d_tags_itr->value); + sout << std::setw(10) << "Offset: " << d_tags_itr->offset + << std::setw(10) << "Source: " << pmt::pmt_symbol_to_string(d_tags_itr->srcid) + << std::setw(10) << "Key: " << pmt::pmt_symbol_to_string(d_tags_itr->key) + << std::setw(10) << "Value: "; + sout << d_tags_itr->value << std::endl; } } } if(d_display) { - std::cout << "----------------------------------------------------------------------"; - std::cout << std::endl; + sout << "----------------------------------------------------------------------"; + sout << std::endl; + + if(d_tags.size() > 0) + std::cout << sout.str(); } return noutput_items; diff --git a/gnuradio-core/src/lib/io/gr_file_sink_base.cc b/gnuradio-core/src/lib/io/gr_file_sink_base.cc index 2dd896ae7e..d0aca418e7 100644 --- a/gnuradio-core/src/lib/io/gr_file_sink_base.cc +++ b/gnuradio-core/src/lib/io/gr_file_sink_base.cc @@ -79,7 +79,6 @@ gr_file_sink_base::open(const char *filename) perror (filename); return false; } - if (d_new_fp){ // if we've already got a new one open, close it fclose(d_new_fp); d_new_fp = 0; diff --git a/gr-blocks/CMakeLists.txt b/gr-blocks/CMakeLists.txt index bbc498c4c0..1fab5d90e0 100644 --- a/gr-blocks/CMakeLists.txt +++ b/gr-blocks/CMakeLists.txt @@ -86,6 +86,7 @@ if(ENABLE_PYTHON) add_subdirectory(swig) add_subdirectory(grc) add_subdirectory(doc) + add_subdirectory(examples) endif(ENABLE_PYTHON) ######################################################################## diff --git a/gr-blocks/examples/CMakeLists.txt b/gr-blocks/examples/CMakeLists.txt new file mode 100644 index 0000000000..79535daa05 --- /dev/null +++ b/gr-blocks/examples/CMakeLists.txt @@ -0,0 +1,20 @@ +# Copyright 2012 Free Software Foundation, Inc. +# +# This file is part of GNU Radio +# +# GNU Radio is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3, or (at your option) +# any later version. +# +# GNU Radio is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with GNU Radio; see the file COPYING. If not, write to +# the Free Software Foundation, Inc., 51 Franklin Street, +# Boston, MA 02110-1301, USA. + +add_subdirectory(metadata) diff --git a/gr-blocks/examples/metadata/CMakeLists.txt b/gr-blocks/examples/metadata/CMakeLists.txt new file mode 100644 index 0000000000..53a54b9d5a --- /dev/null +++ b/gr-blocks/examples/metadata/CMakeLists.txt @@ -0,0 +1,30 @@ +# Copyright 2012 Free Software Foundation, Inc. +# +# This file is part of GNU Radio +# +# GNU Radio is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3, or (at your option) +# any later version. +# +# GNU Radio is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with GNU Radio; see the file COPYING. If not, write to +# the Free Software Foundation, Inc., 51 Franklin Street, +# Boston, MA 02110-1301, USA. + +include(GrPython) + +install( + FILES + file_metadata_sink.grc + file_metadata_source.grc + file_metadata_vector_sink.grc + file_metadata_vector_source.grc + DESTINATION ${GR_PKG_DATA_DIR}/examples/metadata + COMPONENT "core_python" +) diff --git a/gr-blocks/examples/metadata/file_metadata_sink.grc b/gr-blocks/examples/metadata/file_metadata_sink.grc new file mode 100644 index 0000000000..198b0725f9 --- /dev/null +++ b/gr-blocks/examples/metadata/file_metadata_sink.grc @@ -0,0 +1,951 @@ +<?xml version='1.0' encoding='ASCII'?> +<flow_graph> + <timestamp>Fri Dec 14 17:09:06 2012</timestamp> + <block> + <key>options</key> + <param> + <key>id</key> + <value>file_metadata_sink</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>title</key> + <value></value> + </param> + <param> + <key>author</key> + <value></value> + </param> + <param> + <key>description</key> + <value></value> + </param> + <param> + <key>window_size</key> + <value>1280, 1024</value> + </param> + <param> + <key>generate_options</key> + <value>qt_gui</value> + </param> + <param> + <key>category</key> + <value>Custom</value> + </param> + <param> + <key>run_options</key> + <value>prompt</value> + </param> + <param> + <key>run</key> + <value>True</value> + </param> + <param> + <key>max_nouts</key> + <value>0</value> + </param> + <param> + <key>realtime_scheduling</key> + <value></value> + </param> + <param> + <key>_coordinate</key> + <value>(10, 10)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <block> + <key>gr_sig_source_x</key> + <param> + <key>id</key> + <value>gr_sig_source_x_0</value> + </param> + <param> + <key>_enabled</key> + <value>False</value> + </param> + <param> + <key>type</key> + <value>complex</value> + </param> + <param> + <key>samp_rate</key> + <value>samp_rate</value> + </param> + <param> + <key>waveform</key> + <value>gr.GR_COS_WAVE</value> + </param> + <param> + <key>freq</key> + <value>1000</value> + </param> + <param> + <key>amp</key> + <value>1</value> + </param> + <param> + <key>offset</key> + <value>0</value> + </param> + <param> + <key>_coordinate</key> + <value>(57, 140)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <block> + <key>variable_qtgui_range</key> + <param> + <key>id</key> + <value>qt_samp_rate</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>label</key> + <value></value> + </param> + <param> + <key>value</key> + <value>samp_rate</value> + </param> + <param> + <key>start</key> + <value>200000</value> + </param> + <param> + <key>stop</key> + <value>5000000</value> + </param> + <param> + <key>step</key> + <value>200000</value> + </param> + <param> + <key>widget</key> + <value>counter_slider</value> + </param> + <param> + <key>orient</key> + <value>Qt.Horizontal</value> + </param> + <param> + <key>min_len</key> + <value>200</value> + </param> + <param> + <key>gui_hint</key> + <value></value> + </param> + <param> + <key>_coordinate</key> + <value>(330, 259)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <block> + <key>variable</key> + <param> + <key>id</key> + <value>samp_rate</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>value</key> + <value>200000</value> + </param> + <param> + <key>_coordinate</key> + <value>(208, 11)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <block> + <key>uhd_usrp_source</key> + <param> + <key>id</key> + <value>uhd_usrp_source_0</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>type</key> + <value>fc32</value> + </param> + <param> + <key>otw</key> + <value></value> + </param> + <param> + <key>stream_args</key> + <value></value> + </param> + <param> + <key>dev_addr</key> + <value>addr=192.168.11.2</value> + </param> + <param> + <key>sync</key> + <value></value> + </param> + <param> + <key>clock_rate</key> + <value>0.0</value> + </param> + <param> + <key>num_mboards</key> + <value>1</value> + </param> + <param> + <key>clock_source0</key> + <value></value> + </param> + <param> + <key>time_source0</key> + <value></value> + </param> + <param> + <key>sd_spec0</key> + <value></value> + </param> + <param> + <key>clock_source1</key> + <value></value> + </param> + <param> + <key>time_source1</key> + <value></value> + </param> + <param> + <key>sd_spec1</key> + <value></value> + </param> + <param> + <key>clock_source2</key> + <value></value> + </param> + <param> + <key>time_source2</key> + <value></value> + </param> + <param> + <key>sd_spec2</key> + <value></value> + </param> + <param> + <key>clock_source3</key> + <value></value> + </param> + <param> + <key>time_source3</key> + <value></value> + </param> + <param> + <key>sd_spec3</key> + <value></value> + </param> + <param> + <key>clock_source4</key> + <value></value> + </param> + <param> + <key>time_source4</key> + <value></value> + </param> + <param> + <key>sd_spec4</key> + <value></value> + </param> + <param> + <key>clock_source5</key> + <value></value> + </param> + <param> + <key>time_source5</key> + <value></value> + </param> + <param> + <key>sd_spec5</key> + <value></value> + </param> + <param> + <key>clock_source6</key> + <value></value> + </param> + <param> + <key>time_source6</key> + <value></value> + </param> + <param> + <key>sd_spec6</key> + <value></value> + </param> + <param> + <key>clock_source7</key> + <value></value> + </param> + <param> + <key>time_source7</key> + <value></value> + </param> + <param> + <key>sd_spec7</key> + <value></value> + </param> + <param> + <key>nchan</key> + <value>1</value> + </param> + <param> + <key>samp_rate</key> + <value>qt_samp_rate</value> + </param> + <param> + <key>center_freq0</key> + <value>98.9e6</value> + </param> + <param> + <key>gain0</key> + <value>30</value> + </param> + <param> + <key>ant0</key> + <value></value> + </param> + <param> + <key>bw0</key> + <value>0</value> + </param> + <param> + <key>center_freq1</key> + <value>0</value> + </param> + <param> + <key>gain1</key> + <value>0</value> + </param> + <param> + <key>ant1</key> + <value></value> + </param> + <param> + <key>bw1</key> + <value>0</value> + </param> + <param> + <key>center_freq2</key> + <value>0</value> + </param> + <param> + <key>gain2</key> + <value>0</value> + </param> + <param> + <key>ant2</key> + <value></value> + </param> + <param> + <key>bw2</key> + <value>0</value> + </param> + <param> + <key>center_freq3</key> + <value>0</value> + </param> + <param> + <key>gain3</key> + <value>0</value> + </param> + <param> + <key>ant3</key> + <value></value> + </param> + <param> + <key>bw3</key> + <value>0</value> + </param> + <param> + <key>center_freq4</key> + <value>0</value> + </param> + <param> + <key>gain4</key> + <value>0</value> + </param> + <param> + <key>ant4</key> + <value></value> + </param> + <param> + <key>bw4</key> + <value>0</value> + </param> + <param> + <key>center_freq5</key> + <value>0</value> + </param> + <param> + <key>gain5</key> + <value>0</value> + </param> + <param> + <key>ant5</key> + <value></value> + </param> + <param> + <key>bw5</key> + <value>0</value> + </param> + <param> + <key>center_freq6</key> + <value>0</value> + </param> + <param> + <key>gain6</key> + <value>0</value> + </param> + <param> + <key>ant6</key> + <value></value> + </param> + <param> + <key>bw6</key> + <value>0</value> + </param> + <param> + <key>center_freq7</key> + <value>0</value> + </param> + <param> + <key>gain7</key> + <value>0</value> + </param> + <param> + <key>ant7</key> + <value></value> + </param> + <param> + <key>bw7</key> + <value>0</value> + </param> + <param> + <key>center_freq8</key> + <value>0</value> + </param> + <param> + <key>gain8</key> + <value>0</value> + </param> + <param> + <key>ant8</key> + <value></value> + </param> + <param> + <key>bw8</key> + <value>0</value> + </param> + <param> + <key>center_freq9</key> + <value>0</value> + </param> + <param> + <key>gain9</key> + <value>0</value> + </param> + <param> + <key>ant9</key> + <value></value> + </param> + <param> + <key>bw9</key> + <value>0</value> + </param> + <param> + <key>center_freq10</key> + <value>0</value> + </param> + <param> + <key>gain10</key> + <value>0</value> + </param> + <param> + <key>ant10</key> + <value></value> + </param> + <param> + <key>bw10</key> + <value>0</value> + </param> + <param> + <key>center_freq11</key> + <value>0</value> + </param> + <param> + <key>gain11</key> + <value>0</value> + </param> + <param> + <key>ant11</key> + <value></value> + </param> + <param> + <key>bw11</key> + <value>0</value> + </param> + <param> + <key>center_freq12</key> + <value>0</value> + </param> + <param> + <key>gain12</key> + <value>0</value> + </param> + <param> + <key>ant12</key> + <value></value> + </param> + <param> + <key>bw12</key> + <value>0</value> + </param> + <param> + <key>center_freq13</key> + <value>0</value> + </param> + <param> + <key>gain13</key> + <value>0</value> + </param> + <param> + <key>ant13</key> + <value></value> + </param> + <param> + <key>bw13</key> + <value>0</value> + </param> + <param> + <key>center_freq14</key> + <value>0</value> + </param> + <param> + <key>gain14</key> + <value>0</value> + </param> + <param> + <key>ant14</key> + <value></value> + </param> + <param> + <key>bw14</key> + <value>0</value> + </param> + <param> + <key>center_freq15</key> + <value>0</value> + </param> + <param> + <key>gain15</key> + <value>0</value> + </param> + <param> + <key>ant15</key> + <value></value> + </param> + <param> + <key>bw15</key> + <value>0</value> + </param> + <param> + <key>center_freq16</key> + <value>0</value> + </param> + <param> + <key>gain16</key> + <value>0</value> + </param> + <param> + <key>ant16</key> + <value></value> + </param> + <param> + <key>bw16</key> + <value>0</value> + </param> + <param> + <key>center_freq17</key> + <value>0</value> + </param> + <param> + <key>gain17</key> + <value>0</value> + </param> + <param> + <key>ant17</key> + <value></value> + </param> + <param> + <key>bw17</key> + <value>0</value> + </param> + <param> + <key>center_freq18</key> + <value>0</value> + </param> + <param> + <key>gain18</key> + <value>0</value> + </param> + <param> + <key>ant18</key> + <value></value> + </param> + <param> + <key>bw18</key> + <value>0</value> + </param> + <param> + <key>center_freq19</key> + <value>0</value> + </param> + <param> + <key>gain19</key> + <value>0</value> + </param> + <param> + <key>ant19</key> + <value></value> + </param> + <param> + <key>bw19</key> + <value>0</value> + </param> + <param> + <key>center_freq20</key> + <value>0</value> + </param> + <param> + <key>gain20</key> + <value>0</value> + </param> + <param> + <key>ant20</key> + <value></value> + </param> + <param> + <key>bw20</key> + <value>0</value> + </param> + <param> + <key>center_freq21</key> + <value>0</value> + </param> + <param> + <key>gain21</key> + <value>0</value> + </param> + <param> + <key>ant21</key> + <value></value> + </param> + <param> + <key>bw21</key> + <value>0</value> + </param> + <param> + <key>center_freq22</key> + <value>0</value> + </param> + <param> + <key>gain22</key> + <value>0</value> + </param> + <param> + <key>ant22</key> + <value></value> + </param> + <param> + <key>bw22</key> + <value>0</value> + </param> + <param> + <key>center_freq23</key> + <value>0</value> + </param> + <param> + <key>gain23</key> + <value>0</value> + </param> + <param> + <key>ant23</key> + <value></value> + </param> + <param> + <key>bw23</key> + <value>0</value> + </param> + <param> + <key>center_freq24</key> + <value>0</value> + </param> + <param> + <key>gain24</key> + <value>0</value> + </param> + <param> + <key>ant24</key> + <value></value> + </param> + <param> + <key>bw24</key> + <value>0</value> + </param> + <param> + <key>center_freq25</key> + <value>0</value> + </param> + <param> + <key>gain25</key> + <value>0</value> + </param> + <param> + <key>ant25</key> + <value></value> + </param> + <param> + <key>bw25</key> + <value>0</value> + </param> + <param> + <key>center_freq26</key> + <value>0</value> + </param> + <param> + <key>gain26</key> + <value>0</value> + </param> + <param> + <key>ant26</key> + <value></value> + </param> + <param> + <key>bw26</key> + <value>0</value> + </param> + <param> + <key>center_freq27</key> + <value>0</value> + </param> + <param> + <key>gain27</key> + <value>0</value> + </param> + <param> + <key>ant27</key> + <value></value> + </param> + <param> + <key>bw27</key> + <value>0</value> + </param> + <param> + <key>center_freq28</key> + <value>0</value> + </param> + <param> + <key>gain28</key> + <value>0</value> + </param> + <param> + <key>ant28</key> + <value></value> + </param> + <param> + <key>bw28</key> + <value>0</value> + </param> + <param> + <key>center_freq29</key> + <value>0</value> + </param> + <param> + <key>gain29</key> + <value>0</value> + </param> + <param> + <key>ant29</key> + <value></value> + </param> + <param> + <key>bw29</key> + <value>0</value> + </param> + <param> + <key>center_freq30</key> + <value>0</value> + </param> + <param> + <key>gain30</key> + <value>0</value> + </param> + <param> + <key>ant30</key> + <value></value> + </param> + <param> + <key>bw30</key> + <value>0</value> + </param> + <param> + <key>center_freq31</key> + <value>0</value> + </param> + <param> + <key>gain31</key> + <value>0</value> + </param> + <param> + <key>ant31</key> + <value></value> + </param> + <param> + <key>bw31</key> + <value>0</value> + </param> + <param> + <key>_coordinate</key> + <value>(53, 269)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <block> + <key>gr_head</key> + <param> + <key>id</key> + <value>gr_head_0</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>type</key> + <value>complex</value> + </param> + <param> + <key>num_items</key> + <value>20000000</value> + </param> + <param> + <key>vlen</key> + <value>1</value> + </param> + <param> + <key>_coordinate</key> + <value>(309, 172)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <block> + <key>blocks_file_meta_sink</key> + <param> + <key>id</key> + <value>blocks_file_meta_sink_0</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>file</key> + <value>/tmp/metadat_file.out</value> + </param> + <param> + <key>type</key> + <value>complex</value> + </param> + <param> + <key>samp_rate</key> + <value>samp_rate</value> + </param> + <param> + <key>rel_rate</key> + <value>1</value> + </param> + <param> + <key>vlen</key> + <value>1</value> + </param> + <param> + <key>max_seg_size</key> + <value>1000000</value> + </param> + <param> + <key>extra_dict</key> + <value>""</value> + </param> + <param> + <key>detached</key> + <value>True</value> + </param> + <param> + <key>unbuffered</key> + <value>False</value> + </param> + <param> + <key>_coordinate</key> + <value>(569, 124)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <connection> + <source_block_id>gr_sig_source_x_0</source_block_id> + <sink_block_id>gr_head_0</sink_block_id> + <source_key>0</source_key> + <sink_key>0</sink_key> + </connection> + <connection> + <source_block_id>uhd_usrp_source_0</source_block_id> + <sink_block_id>gr_head_0</sink_block_id> + <source_key>0</source_key> + <sink_key>0</sink_key> + </connection> + <connection> + <source_block_id>gr_head_0</source_block_id> + <sink_block_id>blocks_file_meta_sink_0</sink_block_id> + <source_key>0</source_key> + <sink_key>0</sink_key> + </connection> +</flow_graph> diff --git a/gr-blocks/examples/metadata/file_metadata_source.grc b/gr-blocks/examples/metadata/file_metadata_source.grc new file mode 100644 index 0000000000..23757881bc --- /dev/null +++ b/gr-blocks/examples/metadata/file_metadata_source.grc @@ -0,0 +1,350 @@ +<?xml version='1.0' encoding='ASCII'?> +<flow_graph> + <timestamp>Fri Dec 14 17:08:09 2012</timestamp> + <block> + <key>options</key> + <param> + <key>id</key> + <value>file_metadata_source</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>title</key> + <value></value> + </param> + <param> + <key>author</key> + <value></value> + </param> + <param> + <key>description</key> + <value></value> + </param> + <param> + <key>window_size</key> + <value>1280, 1024</value> + </param> + <param> + <key>generate_options</key> + <value>qt_gui</value> + </param> + <param> + <key>category</key> + <value>Custom</value> + </param> + <param> + <key>run_options</key> + <value>prompt</value> + </param> + <param> + <key>run</key> + <value>True</value> + </param> + <param> + <key>max_nouts</key> + <value>0</value> + </param> + <param> + <key>realtime_scheduling</key> + <value></value> + </param> + <param> + <key>_coordinate</key> + <value>(10, 10)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <block> + <key>variable</key> + <param> + <key>id</key> + <value>samp_rate</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>value</key> + <value>200000</value> + </param> + <param> + <key>_coordinate</key> + <value>(208, 11)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <block> + <key>gr_tag_debug</key> + <param> + <key>id</key> + <value>gr_tag_debug_0</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>type</key> + <value>complex</value> + </param> + <param> + <key>name</key> + <value>Test</value> + </param> + <param> + <key>num_inputs</key> + <value>1</value> + </param> + <param> + <key>vlen</key> + <value>1</value> + </param> + <param> + <key>display</key> + <value>True</value> + </param> + <param> + <key>_coordinate</key> + <value>(561, 290)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <block> + <key>analog_agc2_xx</key> + <param> + <key>id</key> + <value>analog_agc2_xx_0</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>type</key> + <value>complex</value> + </param> + <param> + <key>attack_rate</key> + <value>1e-1</value> + </param> + <param> + <key>decay_rate</key> + <value>1e-2</value> + </param> + <param> + <key>reference</key> + <value>1.0</value> + </param> + <param> + <key>gain</key> + <value>1.0</value> + </param> + <param> + <key>max_gain</key> + <value>0.0</value> + </param> + <param> + <key>_coordinate</key> + <value>(561, 82)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <block> + <key>qtgui_time_sink_x</key> + <param> + <key>id</key> + <value>qtgui_time_sink_x_0</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>type</key> + <value>complex</value> + </param> + <param> + <key>name</key> + <value>QT GUI Plot</value> + </param> + <param> + <key>size</key> + <value>1024</value> + </param> + <param> + <key>bw</key> + <value>samp_rate</value> + </param> + <param> + <key>nconnections</key> + <value>1</value> + </param> + <param> + <key>gui_hint</key> + <value></value> + </param> + <param> + <key>_coordinate</key> + <value>(776, 98)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <block> + <key>gr_file_sink</key> + <param> + <key>id</key> + <value>gr_file_sink_1</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>file</key> + <value>/tmp/received_data.out</value> + </param> + <param> + <key>type</key> + <value>complex</value> + </param> + <param> + <key>vlen</key> + <value>1</value> + </param> + <param> + <key>unbuffered</key> + <value>False</value> + </param> + <param> + <key>_coordinate</key> + <value>(564, 215)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <block> + <key>gr_throttle</key> + <param> + <key>id</key> + <value>gr_throttle_0</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>type</key> + <value>complex</value> + </param> + <param> + <key>samples_per_second</key> + <value>samp_rate</value> + </param> + <param> + <key>vlen</key> + <value>1</value> + </param> + <param> + <key>_coordinate</key> + <value>(322, 222)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <block> + <key>blocks_file_meta_source</key> + <param> + <key>id</key> + <value>blocks_file_meta_source_0</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>type</key> + <value>complex</value> + </param> + <param> + <key>file</key> + <value>/tmp/metadat_file.out</value> + </param> + <param> + <key>repeat</key> + <value>False</value> + </param> + <param> + <key>detached</key> + <value>True</value> + </param> + <param> + <key>hdr_file</key> + <value></value> + </param> + <param> + <key>vlen</key> + <value>1</value> + </param> + <param> + <key>_coordinate</key> + <value>(50, 198)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <connection> + <source_block_id>gr_throttle_0</source_block_id> + <sink_block_id>gr_file_sink_1</sink_block_id> + <source_key>0</source_key> + <sink_key>0</sink_key> + </connection> + <connection> + <source_block_id>gr_throttle_0</source_block_id> + <sink_block_id>gr_tag_debug_0</sink_block_id> + <source_key>0</source_key> + <sink_key>0</sink_key> + </connection> + <connection> + <source_block_id>gr_throttle_0</source_block_id> + <sink_block_id>analog_agc2_xx_0</sink_block_id> + <source_key>0</source_key> + <sink_key>0</sink_key> + </connection> + <connection> + <source_block_id>analog_agc2_xx_0</source_block_id> + <sink_block_id>qtgui_time_sink_x_0</sink_block_id> + <source_key>0</source_key> + <sink_key>0</sink_key> + </connection> + <connection> + <source_block_id>blocks_file_meta_source_0</source_block_id> + <sink_block_id>gr_throttle_0</sink_block_id> + <source_key>0</source_key> + <sink_key>0</sink_key> + </connection> +</flow_graph> diff --git a/gr-blocks/examples/metadata/file_metadata_vector_sink.grc b/gr-blocks/examples/metadata/file_metadata_vector_sink.grc new file mode 100644 index 0000000000..05b7cbc922 --- /dev/null +++ b/gr-blocks/examples/metadata/file_metadata_vector_sink.grc @@ -0,0 +1,219 @@ +<?xml version='1.0' encoding='ASCII'?> +<flow_graph> + <timestamp>Fri Dec 14 17:08:29 2012</timestamp> + <block> + <key>options</key> + <param> + <key>id</key> + <value>file_metadata_vector_sink</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>title</key> + <value></value> + </param> + <param> + <key>author</key> + <value></value> + </param> + <param> + <key>description</key> + <value></value> + </param> + <param> + <key>window_size</key> + <value>1280, 1024</value> + </param> + <param> + <key>generate_options</key> + <value>no_gui</value> + </param> + <param> + <key>category</key> + <value>Custom</value> + </param> + <param> + <key>run_options</key> + <value>run</value> + </param> + <param> + <key>run</key> + <value>True</value> + </param> + <param> + <key>max_nouts</key> + <value>0</value> + </param> + <param> + <key>realtime_scheduling</key> + <value></value> + </param> + <param> + <key>_coordinate</key> + <value>(10, 10)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <block> + <key>variable</key> + <param> + <key>id</key> + <value>samp_rate</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>value</key> + <value>200000</value> + </param> + <param> + <key>_coordinate</key> + <value>(208, 11)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <block> + <key>gr_vector_source_x</key> + <param> + <key>id</key> + <value>gr_vector_source_x_0</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>type</key> + <value>complex</value> + </param> + <param> + <key>vector</key> + <value>10*[0,1,2,3,4,5,6,7,8,9]</value> + </param> + <param> + <key>repeat</key> + <value>True</value> + </param> + <param> + <key>vlen</key> + <value>10</value> + </param> + <param> + <key>_coordinate</key> + <value>(67, 100)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <block> + <key>gr_head</key> + <param> + <key>id</key> + <value>gr_head_0</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>type</key> + <value>complex</value> + </param> + <param> + <key>num_items</key> + <value>10010000</value> + </param> + <param> + <key>vlen</key> + <value>10</value> + </param> + <param> + <key>_coordinate</key> + <value>(325, 108)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <block> + <key>blocks_file_meta_sink</key> + <param> + <key>id</key> + <value>blocks_file_meta_sink_0</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>file</key> + <value>/tmp/metadat_vector.out</value> + </param> + <param> + <key>type</key> + <value>complex</value> + </param> + <param> + <key>samp_rate</key> + <value>samp_rate</value> + </param> + <param> + <key>rel_rate</key> + <value>1</value> + </param> + <param> + <key>vlen</key> + <value>10</value> + </param> + <param> + <key>max_seg_size</key> + <value>1000000</value> + </param> + <param> + <key>extra_dict</key> + <value>""</value> + </param> + <param> + <key>detached</key> + <value>True</value> + </param> + <param> + <key>unbuffered</key> + <value>False</value> + </param> + <param> + <key>_coordinate</key> + <value>(544, 60)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <connection> + <source_block_id>gr_vector_source_x_0</source_block_id> + <sink_block_id>gr_head_0</sink_block_id> + <source_key>0</source_key> + <sink_key>0</sink_key> + </connection> + <connection> + <source_block_id>gr_head_0</source_block_id> + <sink_block_id>blocks_file_meta_sink_0</sink_block_id> + <source_key>0</source_key> + <sink_key>0</sink_key> + </connection> +</flow_graph> diff --git a/gr-blocks/examples/metadata/file_metadata_vector_source.grc b/gr-blocks/examples/metadata/file_metadata_vector_source.grc new file mode 100644 index 0000000000..d52257e06d --- /dev/null +++ b/gr-blocks/examples/metadata/file_metadata_vector_source.grc @@ -0,0 +1,338 @@ +<?xml version='1.0' encoding='ASCII'?> +<flow_graph> + <timestamp>Fri Dec 14 17:11:02 2012</timestamp> + <block> + <key>options</key> + <param> + <key>id</key> + <value>file_metadata_vector_source</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>title</key> + <value></value> + </param> + <param> + <key>author</key> + <value></value> + </param> + <param> + <key>description</key> + <value></value> + </param> + <param> + <key>window_size</key> + <value>1280, 1024</value> + </param> + <param> + <key>generate_options</key> + <value>qt_gui</value> + </param> + <param> + <key>category</key> + <value>Custom</value> + </param> + <param> + <key>run_options</key> + <value>prompt</value> + </param> + <param> + <key>run</key> + <value>True</value> + </param> + <param> + <key>max_nouts</key> + <value>0</value> + </param> + <param> + <key>realtime_scheduling</key> + <value></value> + </param> + <param> + <key>_coordinate</key> + <value>(10, 10)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <block> + <key>variable</key> + <param> + <key>id</key> + <value>samp_rate</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>value</key> + <value>200000</value> + </param> + <param> + <key>_coordinate</key> + <value>(208, 11)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <block> + <key>gr_file_sink</key> + <param> + <key>id</key> + <value>gr_file_sink_1</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>file</key> + <value>/tmp/received_data.out</value> + </param> + <param> + <key>type</key> + <value>complex</value> + </param> + <param> + <key>vlen</key> + <value>10</value> + </param> + <param> + <key>unbuffered</key> + <value>False</value> + </param> + <param> + <key>_coordinate</key> + <value>(564, 215)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <block> + <key>gr_tag_debug</key> + <param> + <key>id</key> + <value>gr_tag_debug_0</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>type</key> + <value>complex</value> + </param> + <param> + <key>name</key> + <value>Test</value> + </param> + <param> + <key>num_inputs</key> + <value>1</value> + </param> + <param> + <key>vlen</key> + <value>10</value> + </param> + <param> + <key>display</key> + <value>True</value> + </param> + <param> + <key>_coordinate</key> + <value>(563, 298)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <block> + <key>qtgui_time_sink_x</key> + <param> + <key>id</key> + <value>qtgui_time_sink_x_0</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>type</key> + <value>complex</value> + </param> + <param> + <key>name</key> + <value>QT GUI Plot</value> + </param> + <param> + <key>size</key> + <value>1024</value> + </param> + <param> + <key>bw</key> + <value>samp_rate</value> + </param> + <param> + <key>nconnections</key> + <value>1</value> + </param> + <param> + <key>gui_hint</key> + <value></value> + </param> + <param> + <key>_coordinate</key> + <value>(746, 116)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <block> + <key>blocks_vector_to_stream</key> + <param> + <key>id</key> + <value>blocks_vector_to_stream_0</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>type</key> + <value>complex</value> + </param> + <param> + <key>num_items</key> + <value>10</value> + </param> + <param> + <key>vlen</key> + <value>1</value> + </param> + <param> + <key>_coordinate</key> + <value>(526, 132)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <block> + <key>gr_throttle</key> + <param> + <key>id</key> + <value>gr_throttle_0</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>type</key> + <value>complex</value> + </param> + <param> + <key>samples_per_second</key> + <value>samp_rate</value> + </param> + <param> + <key>vlen</key> + <value>10</value> + </param> + <param> + <key>_coordinate</key> + <value>(322, 223)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <block> + <key>blocks_file_meta_source</key> + <param> + <key>id</key> + <value>blocks_file_meta_source_0</value> + </param> + <param> + <key>_enabled</key> + <value>True</value> + </param> + <param> + <key>type</key> + <value>complex</value> + </param> + <param> + <key>file</key> + <value>/tmp/metadat_vector.out</value> + </param> + <param> + <key>repeat</key> + <value>False</value> + </param> + <param> + <key>detached</key> + <value>True</value> + </param> + <param> + <key>hdr_file</key> + <value></value> + </param> + <param> + <key>vlen</key> + <value>10</value> + </param> + <param> + <key>_coordinate</key> + <value>(42, 199)</value> + </param> + <param> + <key>_rotation</key> + <value>0</value> + </param> + </block> + <connection> + <source_block_id>gr_throttle_0</source_block_id> + <sink_block_id>gr_file_sink_1</sink_block_id> + <source_key>0</source_key> + <sink_key>0</sink_key> + </connection> + <connection> + <source_block_id>gr_throttle_0</source_block_id> + <sink_block_id>gr_tag_debug_0</sink_block_id> + <source_key>0</source_key> + <sink_key>0</sink_key> + </connection> + <connection> + <source_block_id>blocks_vector_to_stream_0</source_block_id> + <sink_block_id>qtgui_time_sink_x_0</sink_block_id> + <source_key>0</source_key> + <sink_key>0</sink_key> + </connection> + <connection> + <source_block_id>gr_throttle_0</source_block_id> + <sink_block_id>blocks_vector_to_stream_0</sink_block_id> + <source_key>0</source_key> + <sink_key>0</sink_key> + </connection> + <connection> + <source_block_id>blocks_file_meta_source_0</source_block_id> + <sink_block_id>gr_throttle_0</sink_block_id> + <source_key>0</source_key> + <sink_key>0</sink_key> + </connection> +</flow_graph> diff --git a/gr-blocks/grc/blocks_block_tree.xml b/gr-blocks/grc/blocks_block_tree.xml index ea5e552d1d..797b3dbc5a 100644 --- a/gr-blocks/grc/blocks_block_tree.xml +++ b/gr-blocks/grc/blocks_block_tree.xml @@ -31,6 +31,11 @@ <cat> <name>Sources (New)</name> <block>blocks_file_source</block> + <block>blocks_file_meta_source</block> + </cat> + <cat> + <name>Sinks (New)</name> + <block>blocks_file_meta_sink</block> </cat> <cat> <name>Math Operations (New) </name> diff --git a/gr-blocks/grc/blocks_file_meta_sink.xml b/gr-blocks/grc/blocks_file_meta_sink.xml new file mode 100644 index 0000000000..f6490d507d --- /dev/null +++ b/gr-blocks/grc/blocks_file_meta_sink.xml @@ -0,0 +1,124 @@ +<?xml version="1.0"?> +<!-- +################################################### +##File Meta Sink +################################################### + --> +<block> + <name>File Meta Sink</name> + <key>blocks_file_meta_sink</key> + <import>from gnuradio import gr, blocks</import> + <make>blocks.file_meta_sink($type.size*$vlen, $file, $samp_rate, $rel_rate, $type.dtype, $type.cplx, $max_seg_size, $extra_dict, $detached) +self.$(id).set_unbuffered($unbuffered)</make> + <callback>set_unbuffered($unbuffered)</callback> + <callback>open($file)</callback> + <param> + <name>File</name> + <key>file</key> + <value></value> + <type>file_save</type> + </param> + <param> + <name>Input Type</name> + <key>type</key> + <type>enum</type> + <option> + <name>Complex</name> + <key>complex</key> + <opt>size:gr.sizeof_gr_complex</opt> + <opt>dtype:blocks.GR_FILE_FLOAT</opt> + <opt>cplx:True</opt> + </option> + <option> + <name>Float</name> + <key>float</key> + <opt>size:gr.sizeof_float</opt> + <opt>dtype:blocks.GR_FILE_FLOAT</opt> + <opt>cplx:False</opt> + </option> + <option> + <name>Int</name> + <key>int</key> + <opt>size:gr.sizeof_int</opt> + <opt>dtype:blocks.GR_FILE_INT</opt> + <opt>cplx:False</opt> + </option> + <option> + <name>Short</name> + <key>short</key> + <opt>size:gr.sizeof_short</opt> + <opt>dtype:blocks.GR_FILE_SHORT</opt> + <opt>cplx:False</opt> + </option> + <option> + <name>Byte</name> + <key>byte</key> + <opt>size:gr.sizeof_char</opt> + <opt>dtype:blocks.GR_FILE_BYTE</opt> + <opt>cplx:False</opt> + </option> + </param> + <param> + <name>Sample Rate</name> + <key>samp_rate</key> + <value>samp_rate</value> + <type>real</type> + </param> + <param> + <name>Relative Rate Change</name> + <key>rel_rate</key> + <value>1</value> + <type>real</type> + </param> + <param> + <name>Vec Length</name> + <key>vlen</key> + <value>1</value> + <type>int</type> + </param> + <param> + <name>Max Seg. Size</name> + <key>max_seg_size</key> + <value>1000000</value> + <type>int</type> + </param> + <param> + <name>Extra Dict.</name> + <key>extra_dict</key> + <value>""</value> + <type>string</type> + </param> + <param> + <name>Detached</name> + <key>detached</key> + <value>False</value> + <type>bool</type> + <option> + <name>Off</name> + <key>False</key> + </option> + <option> + <name>On</name> + <key>True</key> + </option> + </param> <param> + <name>Unbuffered</name> + <key>unbuffered</key> + <value>False</value> + <type>bool</type> + <option> + <name>Off</name> + <key>False</key> + </option> + <option> + <name>On</name> + <key>True</key> + </option> + </param> + <check>$vlen > 0</check> + <sink> + <name>in</name> + <type>$type</type> + <vlen>$vlen</vlen> + </sink> +</block> diff --git a/gr-blocks/grc/blocks_file_meta_source.xml b/gr-blocks/grc/blocks_file_meta_source.xml new file mode 100644 index 0000000000..aa7e349551 --- /dev/null +++ b/gr-blocks/grc/blocks_file_meta_source.xml @@ -0,0 +1,94 @@ +<?xml version="1.0"?> +<!-- +################################################### +##File Source +################################################### + --> +<block> + <name>File Meta Source</name> + <key>blocks_file_meta_source</key> + <import>from gnuradio import gr, blocks</import> + <make>blocks.file_meta_source($file, $repeat, $detached, $hdr_file)</make> + <callback>open($file, $repeat)</callback> + <param> + <name>Output Type</name> + <key>type</key> + <type>enum</type> + <option> + <name>Complex</name> + <key>complex</key> + <opt>size:gr.sizeof_gr_complex</opt> + </option> + <option> + <name>Float</name> + <key>float</key> + <opt>size:gr.sizeof_float</opt> + </option> + <option> + <name>Int</name> + <key>int</key> + <opt>size:gr.sizeof_int</opt> + </option> + <option> + <name>Short</name> + <key>short</key> + <opt>size:gr.sizeof_short</opt> + </option> + <option> + <name>Byte</name> + <key>byte</key> + <opt>size:gr.sizeof_char</opt> + </option> + </param> + <param> + <name>File</name> + <key>file</key> + <value></value> + <type>file_open</type> + </param> + <param> + <name>Repeat</name> + <key>repeat</key> + <value>True</value> + <type>enum</type> + <option> + <name>Yes</name> + <key>True</key> + </option> + <option> + <name>No</name> + <key>False</key> + </option> + </param> + <param> + <name>Detached Header</name> + <key>detached</key> + <value>False</value> + <type>enum</type> + <option> + <name>Yes</name> + <key>True</key> + </option> + <option> + <name>No</name> + <key>False</key> + </option> + </param> + <param> + <name>Header File</name> + <key>hdr_file</key> + <value></value> + <type>file_open</type> + </param> + <param> + <name>Vec Length</name> + <key>vlen</key> + <value>1</value> + <type>int</type> + </param> + <source> + <name>out</name> + <type>$type</type> + <vlen>$vlen</vlen> + </source> +</block> diff --git a/gr-blocks/include/blocks/CMakeLists.txt b/gr-blocks/include/blocks/CMakeLists.txt index 787fddba37..0459097b0d 100644 --- a/gr-blocks/include/blocks/CMakeLists.txt +++ b/gr-blocks/include/blocks/CMakeLists.txt @@ -102,6 +102,8 @@ install(FILES conjugate_cc.h deinterleave.h file_source.h + file_meta_sink.h + file_meta_source.h float_to_char.h float_to_complex.h float_to_int.h diff --git a/gr-blocks/include/blocks/file_meta_sink.h b/gr-blocks/include/blocks/file_meta_sink.h new file mode 100644 index 0000000000..ef64887cf9 --- /dev/null +++ b/gr-blocks/include/blocks/file_meta_sink.h @@ -0,0 +1,111 @@ +/* -*- c++ -*- */ +/* + * Copyright 2012 Free Software Foundation, Inc. + * + * This file is part of GNU Radio + * + * GNU Radio is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 3, or (at your option) + * any later version. + * + * GNU Radio is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with GNU Radio; see the file COPYING. If not, write to + * the Free Software Foundation, Inc., 51 Franklin Street, + * Boston, MA 02110-1301, USA. + */ + +#ifndef INCLUDED_BLOCKS_FILE_META_SINK_H +#define INCLUDED_BLOCKS_FILE_META_SINK_H + +#include <blocks/api.h> +#include <gr_sync_block.h> + +namespace gr { + namespace blocks { + + const char METADATA_VERSION = 0; + const size_t METADATA_HEADER_SIZE = 149; + + enum gr_file_types { + GR_FILE_BYTE=0, + GR_FILE_CHAR=0, + GR_FILE_SHORT=1, + GR_FILE_INT, + GR_FILE_LONG, + GR_FILE_LONG_LONG, + GR_FILE_FLOAT, + GR_FILE_DOUBLE, + }; + + /*! + * \brief Write stream to file with meta-data headers. + * \ingroup sink_blk + * + * These files represent data as binary information in between + * meta-data headers. The headers contain information about the + * type of data and properties of the data in the next segment of + * samples. The information includes: + * + * rx_rate (double): sample rate of data. + * rx_time (uint64_t, double): time stamp of first sample in segment. + * size (uint32_t): item size in bytes. + * type (gr_file_types as int32_t): data type. + * cplx (bool): Is data complex? + * strt (uint64_t): Starting byte of data in this segment. + * bytes (uint64_t): Size in bytes of data in this segment. + * + * Tags can be sent to the file to update the information, which + * will create a new header. Headers are found by searching from + * the first header (at position 0 in the file) and reading where + * the data segment starts plus the data segment size. Following + * will either be a new header or EOF. + */ + class BLOCKS_API file_meta_sink : virtual public gr_sync_block + { + public: + // gr::blocks::file_meta_sink::sptr + typedef boost::shared_ptr<file_meta_sink> sptr; + + /*! + * \brief Create a meta-data file sink. + * + * \param itemsize (size_t): Size of data type. + * \param filename (string): Name of file to write data to. + * \param samp_rate (double): Sample rate of data. If sample rate will be + * set by a tag, such as rx_tag from a UHD source, this is + * basically ignored. + * \param relative_rate (double): Rate chance from source of sample + * rate tag to sink. + * \param type (gr_file_types): Data type (int, float, etc.) + * \param complex (bool): If data stream is complex + * \param max_segment_size (size_t): Length of a single segment + * before the header is repeated (in items). + * \param extra_dict (string): a serialized PMT dictionary of extra + * information. Currently not supported. + * \param detached_header (bool): Set to true to store the header + * info in a separate file (named filename.hdr) + */ + static sptr make(size_t itemsize, const std::string &filename, + double samp_rate=1, double relative_rate=1, + gr_file_types type=GR_FILE_FLOAT, bool complex=true, + size_t max_segment_size=1000000, + const std::string &extra_dict="", + bool detached_header=false); + + virtual bool open(const std::string &filename) = 0; + virtual void close() = 0; + virtual void do_update() = 0; + + virtual void set_unbuffered(bool unbuffered) = 0; + }; + + } /* namespace blocks */ +} /* namespace gr */ + +#endif /* INCLUDED_BLOCKS_FILE_META_SINK_H */ diff --git a/gr-blocks/include/blocks/file_meta_source.h b/gr-blocks/include/blocks/file_meta_source.h new file mode 100644 index 0000000000..a992d52432 --- /dev/null +++ b/gr-blocks/include/blocks/file_meta_source.h @@ -0,0 +1,81 @@ +/* -*- c++ -*- */ +/* + * Copyright 2012 Free Software Foundation, Inc. + * + * This file is part of GNU Radio + * + * GNU Radio is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 3, or (at your option) + * any later version. + * + * GNU Radio is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with GNU Radio; see the file COPYING. If not, write to + * the Free Software Foundation, Inc., 51 Franklin Street, + * Boston, MA 02110-1301, USA. + */ + +#ifndef INCLUDED_BLOCKS_FILE_META_SOURCE_H +#define INCLUDED_BLOCKS_FILE_META_SOURCE_H + +#include <blocks/api.h> +#include <gr_sync_block.h> + +namespace gr { + namespace blocks { + + /*! + * \brief Reads stream from file with meta-data headers. Headers + * are parsed into tags. + * \ingroup source_blk + * + * The information in the metadata headers includes: + * + * rx_rate (double): sample rate of data. + * rx_time (uint64_t, double): time stamp of first sample in segment. + * size (uint32_t): item size in bytes. + * type (gr_file_types as int32_t): data type. + * cplx (bool): Is data complex? + * strt (uint64_t): Starting byte of data in this segment. + * bytes (uint64_t): Size in bytes of data in this segment. + * + * Any item inside of the extra header dictionary is ready out and + * made into a stream tag. + */ + class BLOCKS_API file_meta_source : virtual public gr_sync_block + { + public: + // gr::blocks::file_meta_source::sptr + typedef boost::shared_ptr<file_meta_source> sptr; + + /*! + * \brief Create a meta-data file source. + * + * \param filename (string): Name of file to write data to. + * \param repeat (bool): Repeats file when EOF is found. + * \param detached_header (bool): Set to true if header + * info is stored in a separate file (usually named filename.hdr) + * \param hdr_filename (string): Name of detached header file if used. + * Defaults to 'filename.hdr' if detached_header is true but this + * field is an empty string. + */ + static sptr make(const std::string &filename, + bool repeat=false, + bool detached_header=false, + const std::string &hdr_filename=""); + + virtual bool open(const std::string &filename, + const std::string &hdr_filename="") = 0; + virtual void close() = 0; + virtual void do_update() = 0; + }; + + } /* namespace blocks */ +} /* namespace gr */ + +#endif /* INCLUDED_BLOCKS_FILE_META_SOURCE_H */ diff --git a/gr-blocks/lib/CMakeLists.txt b/gr-blocks/lib/CMakeLists.txt index 3a8ffac753..ab989fc782 100644 --- a/gr-blocks/lib/CMakeLists.txt +++ b/gr-blocks/lib/CMakeLists.txt @@ -141,6 +141,8 @@ list(APPEND gr_blocks_sources conjugate_cc_impl.cc deinterleave_impl.cc file_source_impl.cc + file_meta_sink_impl.cc + file_meta_source_impl.cc float_to_char_impl.cc float_to_complex_impl.cc float_array_to_int.cc diff --git a/gr-blocks/lib/file_meta_sink_impl.cc b/gr-blocks/lib/file_meta_sink_impl.cc new file mode 100644 index 0000000000..ad16e9fcac --- /dev/null +++ b/gr-blocks/lib/file_meta_sink_impl.cc @@ -0,0 +1,464 @@ +/* -*- c++ -*- */ +/* + * Copyright 2012 Free Software Foundation, Inc. + * + * This file is part of GNU Radio + * + * GNU Radio is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 3, or (at your option) + * any later version. + * + * GNU Radio is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with GNU Radio; see the file COPYING. If not, write to + * the Free Software Foundation, Inc., 51 Franklin Street, + * Boston, MA 02110-1301, USA. + */ + +#ifdef HAVE_CONFIG_H +#include "config.h" +#endif + +#include "file_meta_sink_impl.h" +#include <gr_io_signature.h> +#include <sys/types.h> +#include <sys/stat.h> +#include <fcntl.h> +#include <stdexcept> +#include <cstdio> + +namespace gr { + namespace blocks { + +// win32 (mingw/msvc) specific +#ifdef HAVE_IO_H +#include <io.h> +#endif +#ifdef O_BINARY +#define OUR_O_BINARY O_BINARY +#else +#define OUR_O_BINARY 0 +#endif + +// should be handled via configure +#ifdef O_LARGEFILE +#define OUR_O_LARGEFILE O_LARGEFILE +#else +#define OUR_O_LARGEFILE 0 +#endif + + file_meta_sink::sptr + file_meta_sink::make(size_t itemsize, const std::string &filename, + double samp_rate, double relative_rate, + gr_file_types type, bool complex, + size_t max_segment_size, + const std::string &extra_dict, + bool detached_header) + { + return gnuradio::get_initial_sptr + (new file_meta_sink_impl(itemsize, filename, + samp_rate, relative_rate, + type, complex, + max_segment_size, + extra_dict, + detached_header)); + } + + file_meta_sink_impl::file_meta_sink_impl(size_t itemsize, + const std::string &filename, + double samp_rate, double relative_rate, + gr_file_types type, bool complex, + size_t max_segment_size, + const std::string &extra_dict, + bool detached_header) + : gr_sync_block("file_meta_sink", + gr_make_io_signature(1, 1, itemsize), + gr_make_io_signature(0, 0, 0)), + d_itemsize(itemsize), + d_samp_rate(samp_rate), d_relative_rate(relative_rate), + d_max_seg_size(max_segment_size), d_total_seg_size(0), + d_updated(false), d_unbuffered(false) + { + d_fp = 0; + d_new_fp = 0; + d_hdr_fp = 0; + d_new_hdr_fp = 0; + + if(detached_header == true) + d_state = STATE_DETACHED; + else + d_state = STATE_INLINE; + + if(!open(filename)) + throw std::runtime_error("file_meta_sink: can't open file\n"); + + pmt_t timestamp = pmt_make_tuple(pmt_from_uint64(0), + pmt_from_double(0)); + + // handle extra dictionary + d_extra = pmt_make_dict(); + if(extra_dict.size() > 0) { + pmt_t extras = pmt_deserialize_str(extra_dict); + pmt_t keys = pmt_dict_keys(extras); + pmt_t vals = pmt_dict_values(extras); + size_t nitems = pmt_length(keys); + for(size_t i = 0; i < nitems; i++) { + d_extra = pmt_dict_add(d_extra, + pmt_nth(i, keys), + pmt_nth(i, vals)); + } + } + + d_extra_size = pmt_serialize_str(d_extra).size(); + + d_header = pmt_make_dict(); + d_header = pmt_dict_add(d_header, mp("version"), mp(METADATA_VERSION)); + d_header = pmt_dict_add(d_header, mp("rx_rate"), mp(samp_rate)); + d_header = pmt_dict_add(d_header, mp("rx_time"), timestamp); + d_header = pmt_dict_add(d_header, mp("size"), pmt_from_long(d_itemsize)); + d_header = pmt_dict_add(d_header, mp("type"), pmt_from_long(type)); + d_header = pmt_dict_add(d_header, mp("cplx"), complex ? PMT_T : PMT_F); + d_header = pmt_dict_add(d_header, mp("strt"), pmt_from_uint64(METADATA_HEADER_SIZE+d_extra_size)); + d_header = pmt_dict_add(d_header, mp("bytes"), pmt_from_uint64(0)); + + do_update(); + + if(d_state == STATE_DETACHED) + write_header(d_hdr_fp, d_header, d_extra); + else + write_header(d_fp, d_header, d_extra); + } + + file_meta_sink_impl::~file_meta_sink_impl() + { + close(); + + if(d_fp) { + fclose(d_fp); + d_fp = 0; + } + + if(d_state == STATE_DETACHED) { + if(d_hdr_fp) { + fclose(d_hdr_fp); + d_hdr_fp = 0; + } + } + } + + bool + file_meta_sink_impl::open(const std::string &filename) + { + bool ret = true; + if(d_state == STATE_DETACHED) { + std::string s = filename + ".hdr"; + ret = _open(&d_new_hdr_fp, s.c_str()); + } + + ret = ret && _open(&d_new_fp, filename.c_str()); + d_updated = true; + return ret; + } + + bool + file_meta_sink_impl::_open(FILE **fp, const char *filename) + { + gruel::scoped_lock guard(d_mutex); // hold mutex for duration of this function + + bool ret = true; + int fd; + + if((fd = ::open(filename, + O_WRONLY|O_CREAT|O_TRUNC|OUR_O_LARGEFILE|OUR_O_BINARY, + 0664)) < 0){ + perror(filename); + return false; + } + + if(*fp) { // if we've already got a new one open, close it + fclose(*fp); + fp = 0; + } + + if((*fp = fdopen(fd, "wb")) == NULL) { + perror(filename); + ::close(fd); // don't leak file descriptor if fdopen fails. + } + + ret = fp != 0; + + return ret; + } + + void + file_meta_sink_impl::close() + { + gruel::scoped_lock guard(d_mutex); // hold mutex for duration of this function + update_last_header(); + + if(d_state == STATE_DETACHED) { + if(d_new_hdr_fp) { + fclose(d_new_hdr_fp); + d_new_hdr_fp = 0; + } + } + + if(d_new_fp) { + fclose(d_new_fp); + d_new_fp = 0; + } + d_updated = true; + } + + void + file_meta_sink_impl::do_update() + { + if(d_updated) { + gruel::scoped_lock guard(d_mutex); // hold mutex for duration of this block + if(d_state == STATE_DETACHED) { + if(d_hdr_fp) + fclose(d_hdr_fp); + d_hdr_fp = d_new_hdr_fp; // install new file pointer + d_new_hdr_fp = 0; + } + + if(d_fp) + fclose(d_fp); + d_fp = d_new_fp; // install new file pointer + d_new_fp = 0; + + d_updated = false; + } + } + + void + file_meta_sink_impl::write_header(FILE *fp, pmt_t header, pmt_t extra) + { + std::string header_str = pmt_serialize_str(header); + std::string extra_str = pmt_serialize_str(extra); + + if((header_str.size() != METADATA_HEADER_SIZE) && (extra_str.size() != d_extra_size)) + throw std::runtime_error("file_meta_sink: header or extras is wrong size.\n"); + + size_t nwritten = 0; + while(nwritten < header_str.size()) { + std::string sub = header_str.substr(nwritten); + int count = fwrite(sub.c_str(), sizeof(char), sub.size(), fp); + nwritten += count; + if((count == 0) && (ferror(fp))) { + fclose(fp); + throw std::runtime_error("file_meta_sink: error writing header to file.\n"); + } + } + + nwritten = 0; + while(nwritten < extra_str.size()) { + std::string sub = extra_str.substr(nwritten); + int count = fwrite(sub.c_str(), sizeof(char), sub.size(), fp); + nwritten += count; + if((count == 0) && (ferror(fp))) { + fclose(fp); + throw std::runtime_error("file_meta_sink: error writing extra to file.\n"); + } + } + + fflush(fp); + } + + void + file_meta_sink_impl::update_header(pmt_t key, pmt_t value) + { + // Special handling caveat to transform rate from radio source into + // the rate at this sink. + if(pmt_eq(key, mp("rx_rate"))) { + d_samp_rate = pmt_to_double(value); + value = pmt_from_double(d_samp_rate*d_relative_rate); + } + + // If the tag is not part of the standard header, we put it into the + // extra data, which either updates the current dictionary or adds a + // new item. + if(pmt_dict_has_key(d_header, key)) { + d_header = pmt_dict_add(d_header, key, value); + } + else { + d_extra = pmt_dict_add(d_extra, key, value); + d_extra_size = pmt_serialize_str(d_extra).size(); + } + } + + void + file_meta_sink_impl::update_last_header() + { + if(d_state == STATE_DETACHED) + update_last_header_detached(); + else + update_last_header_inline(); + } + + void + file_meta_sink_impl::update_last_header_inline() + { + // Update the last header info with the number of samples this + // block represents. + + size_t hdrlen = pmt_to_uint64(pmt_dict_ref(d_header, mp("strt"), PMT_NIL)); + size_t seg_size = d_itemsize*d_total_seg_size; + pmt_t s = pmt_from_uint64(seg_size); + update_header(mp("bytes"), s); + update_header(mp("strt"), pmt_from_uint64(METADATA_HEADER_SIZE+d_extra_size)); + fseek(d_fp, -seg_size-hdrlen, SEEK_CUR); + write_header(d_fp, d_header, d_extra); + fseek(d_fp, seg_size, SEEK_CUR); + } + + void + file_meta_sink_impl::update_last_header_detached() + { + // Update the last header info with the number of samples this + // block represents. + size_t hdrlen = pmt_to_uint64(pmt_dict_ref(d_header, mp("strt"), PMT_NIL)); + size_t seg_size = d_itemsize*d_total_seg_size; + pmt_t s = pmt_from_uint64(seg_size); + update_header(mp("bytes"), s); + update_header(mp("strt"), pmt_from_uint64(METADATA_HEADER_SIZE+d_extra_size)); + fseek(d_hdr_fp, -hdrlen, SEEK_CUR); + write_header(d_hdr_fp, d_header, d_extra); + } + + void + file_meta_sink_impl::write_and_update() + { + // New header, so set current size of chunk to 0 and start of chunk + // based on current index + header size. + //uint64_t loc = get_last_header_loc(); + pmt_t s = pmt_from_uint64(0); + update_header(mp("bytes"), s); + + // If we have multiple tags on the same offset, this makes + // sure we just overwrite the same header each time instead + // of creating a new header per tag. + s = pmt_from_uint64(METADATA_HEADER_SIZE + d_extra_size); + update_header(mp("strt"), s); + + if(d_state == STATE_DETACHED) + write_header(d_hdr_fp, d_header, d_extra); + else + write_header(d_fp, d_header, d_extra); + } + + void + file_meta_sink_impl::update_rx_time() + { + pmt_t rx_time = pmt_string_to_symbol("rx_time"); + pmt_t r = pmt_dict_ref(d_header, rx_time, PMT_NIL); + uint64_t secs = pmt_to_uint64(pmt_tuple_ref(r, 0)); + double fracs = pmt_to_double(pmt_tuple_ref(r, 1)); + double diff = d_total_seg_size / (d_samp_rate*d_relative_rate); + + //std::cerr << "old secs: " << secs << std::endl; + //std::cerr << "old fracs: " << fracs << std::endl; + //std::cerr << "seg size: " << d_total_seg_size << std::endl; + //std::cerr << "diff: " << diff << std::endl; + + fracs += diff; + uint64_t new_secs = static_cast<uint64_t>(fracs); + secs += new_secs; + fracs -= new_secs; + + //std::cerr << "new secs: " << secs << std::endl; + //std::cerr << "new fracs: " << fracs << std::endl << std::endl; + + r = pmt_make_tuple(pmt_from_uint64(secs), pmt_from_double(fracs)); + d_header = pmt_dict_add(d_header, rx_time, r); + } + + int + file_meta_sink_impl::work(int noutput_items, + gr_vector_const_void_star &input_items, + gr_vector_void_star &output_items) + { + char *inbuf = (char*)input_items[0]; + int nwritten = 0; + + do_update(); // update d_fp is reqd + + if(!d_fp) + return noutput_items; // drop output on the floor + + uint64_t abs_N = nitems_read(0); + uint64_t end_N = abs_N + (uint64_t)(noutput_items); + std::vector<gr_tag_t> all_tags; + get_tags_in_range(all_tags, 0, abs_N, end_N); + + std::vector<gr_tag_t>::iterator itr; + for(itr = all_tags.begin(); itr != all_tags.end(); itr++) { + int item_offset = (int)(itr->offset - abs_N); + + // Write date to file up to the next tag location + while(nwritten < item_offset) { + size_t towrite = std::min(d_max_seg_size - d_total_seg_size, + (size_t)(item_offset - nwritten)); + int count = fwrite(inbuf, d_itemsize, towrite, d_fp); + if(count == 0) // FIXME add error handling + break; + nwritten += count; + inbuf += count * d_itemsize; + + d_total_seg_size += count; + + // Only add a new header if we are not at the position of the + // next tag + if((d_total_seg_size == d_max_seg_size) && + (nwritten < item_offset)) { + update_last_header(); + update_rx_time(); + write_and_update(); + d_total_seg_size = 0; + } + } + + if(d_total_seg_size > 0) { + update_last_header(); + update_header(itr->key, itr->value); + write_and_update(); + d_total_seg_size = 0; + } + else { + update_header(itr->key, itr->value); + update_last_header(); + } + } + + // Finish up the rest of the data after tags + while(nwritten < noutput_items) { + size_t towrite = std::min(d_max_seg_size - d_total_seg_size, + (size_t)(noutput_items - nwritten)); + int count = fwrite(inbuf, d_itemsize, towrite, d_fp); + if(count == 0) // FIXME add error handling + break; + nwritten += count; + inbuf += count * d_itemsize; + + d_total_seg_size += count; + if(d_total_seg_size == d_max_seg_size) { + update_last_header(); + update_rx_time(); + write_and_update(); + d_total_seg_size = 0; + } + } + + if(d_unbuffered) + fflush(d_fp); + + return nwritten; + } + + } /* namespace blocks */ +} /* namespace gr */ diff --git a/gr-blocks/lib/file_meta_sink_impl.h b/gr-blocks/lib/file_meta_sink_impl.h new file mode 100644 index 0000000000..566c997b3d --- /dev/null +++ b/gr-blocks/lib/file_meta_sink_impl.h @@ -0,0 +1,96 @@ +/* -*- c++ -*- */ +/* + * Copyright 2012 Free Software Foundation, Inc. + * + * This file is part of GNU Radio + * + * GNU Radio is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 3, or (at your option) + * any later version. + * + * GNU Radio is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with GNU Radio; see the file COPYING. If not, write to + * the Free Software Foundation, Inc., 51 Franklin Street, + * Boston, MA 02110-1301, USA. + */ + +#ifndef INCLUDED_BLOCKS_FILE_META_SINK_IMPL_H +#define INCLUDED_BLOCKS_FILE_META_SINK_IMPL_H + +#include <blocks/file_meta_sink.h> +#include <gruel/pmt.h> +#include <gruel/thread.h> + +using namespace pmt; + +namespace gr { + namespace blocks { + + class file_meta_sink_impl : public file_meta_sink + { + private: + enum meta_state_t { + STATE_INLINE=0, + STATE_DETACHED + }; + + size_t d_itemsize; + double d_samp_rate; + double d_relative_rate; + size_t d_max_seg_size; + size_t d_total_seg_size; + pmt_t d_header; + pmt_t d_extra; + size_t d_extra_size; + bool d_updated; + bool d_unbuffered; + + boost::mutex d_mutex; + FILE *d_new_fp, *d_new_hdr_fp; + FILE *d_fp, *d_hdr_fp; + meta_state_t d_state; + + protected: + void write_header(FILE *fp, pmt_t header, pmt_t extra); + void update_header(pmt_t key, pmt_t value); + void update_last_header(); + void update_last_header_inline(); + void update_last_header_detached(); + void write_and_update(); + void update_rx_time(); + + bool _open(FILE **fp, const char *filename); + + public: + file_meta_sink_impl(size_t itemsize, const std::string &filename, + double samp_rate=1, double relative_rate=1, + gr_file_types type=GR_FILE_FLOAT, bool complex=true, + size_t max_segment_size=1000000, + const std::string &extra_dict="", + bool detached_header=false); + ~file_meta_sink_impl(); + + bool open(const std::string &filename); + void close(); + void do_update(); + + void set_unbuffered(bool unbuffered) + { + d_unbuffered = unbuffered; + } + + int work(int noutput_items, + gr_vector_const_void_star &input_items, + gr_vector_void_star &output_items); + }; + + } /* namespace blocks */ +} /* namespace gr */ + +#endif /* INCLUDED_BLOCKS_FILE_META_SINK_IMPL_H */ diff --git a/gr-blocks/lib/file_meta_source_impl.cc b/gr-blocks/lib/file_meta_source_impl.cc new file mode 100644 index 0000000000..fb39b205b4 --- /dev/null +++ b/gr-blocks/lib/file_meta_source_impl.cc @@ -0,0 +1,433 @@ +/* -*- c++ -*- */ +/* + * Copyright 2012 Free Software Foundation, Inc. + * + * This file is part of GNU Radio + * + * GNU Radio is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 3, or (at your option) + * any later version. + * + * GNU Radio is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with GNU Radio; see the file COPYING. If not, write to + * the Free Software Foundation, Inc., 51 Franklin Street, + * Boston, MA 02110-1301, USA. + */ + +#ifdef HAVE_CONFIG_H +#include "config.h" +#endif + +#include "file_meta_source_impl.h" +#include <gr_io_signature.h> +#include <sys/types.h> +#include <sys/stat.h> +#include <fcntl.h> +#include <stdexcept> +#include <cstdio> + +namespace gr { + namespace blocks { + +// win32 (mingw/msvc) specific +#ifdef HAVE_IO_H +#include <io.h> +#endif +#ifdef O_BINARY +#define OUR_O_BINARY O_BINARY +#else +#define OUR_O_BINARY 0 +#endif + +// should be handled via configure +#ifdef O_LARGEFILE +#define OUR_O_LARGEFILE O_LARGEFILE +#else +#define OUR_O_LARGEFILE 0 +#endif + + file_meta_source::sptr + file_meta_source::make(const std::string &filename, + bool repeat, + bool detached_header, + const std::string &hdr_filename) + { + return gnuradio::get_initial_sptr + (new file_meta_source_impl(filename, + repeat, + detached_header, + hdr_filename)); + } + + file_meta_source_impl::file_meta_source_impl(const std::string &filename, + bool repeat, + bool detached_header, + const std::string &hdr_filename) + : gr_sync_block("file_meta_source", + gr_make_io_signature(0, 0, 0), + gr_make_io_signature(1, 1, 1)), + d_itemsize(0), d_samp_rate(0), + d_seg_size(0), + d_updated(false), d_repeat(repeat) + { + d_fp = 0; + d_new_fp = 0; + d_hdr_fp = 0; + d_new_hdr_fp = 0; + + if(detached_header == true) { + d_state = STATE_DETACHED; + } + else + d_state = STATE_INLINE; + + if(!open(filename, hdr_filename)) + throw std::runtime_error("file_meta_source: can't open file\n"); + + do_update(); + + pmt_t hdr = PMT_NIL, extras = PMT_NIL; + if(read_header(hdr, extras)) { + parse_header(hdr, 0, d_tags); + parse_extras(extras, 0, d_tags); + } + else + throw std::runtime_error("file_meta_source: could not read header.\n"); + + // Set output signature based on itemsize info in header + set_output_signature(gr_make_io_signature(1, 1, d_itemsize)); + } + + file_meta_source_impl::~file_meta_source_impl() + { + close(); + + if(d_fp) { + fclose(d_fp); + d_fp = 0; + } + + if(d_state == STATE_DETACHED) { + if(d_hdr_fp) { + fclose(d_hdr_fp); + d_hdr_fp = 0; + } + } + } + + bool + file_meta_source_impl::read_header(pmt_t &hdr, pmt_t &extras) + { + // Select which file handle to read from. + FILE *fp; + if(d_state == STATE_DETACHED) + fp = d_hdr_fp; + else + fp = d_fp; + + size_t ret; + size_t size = 0; + std::string str; + char *hdr_buffer = new char[METADATA_HEADER_SIZE]; + while(size < METADATA_HEADER_SIZE) { + ret = fread(&hdr_buffer[size], sizeof(char), METADATA_HEADER_SIZE-size, fp); + if(ret == 0) { + delete [] hdr_buffer; + if(feof(fp)) + return false; + else { + std::stringstream s; + s << "file_meta_source: error occurred extracting header: " + << strerror(errno) << std::endl; + throw std::runtime_error(s.str()); + } + } + size += ret; + } + + // Convert to string or the char array gets confused by the \0 + str.insert(0, hdr_buffer, METADATA_HEADER_SIZE); + hdr = pmt_deserialize_str(str); + delete [] hdr_buffer; + + uint64_t seg_start, extra_len; + pmt_t r, dump; + if(pmt_dict_has_key(hdr, pmt_string_to_symbol("strt"))) { + r = pmt_dict_ref(hdr, pmt_string_to_symbol("strt"), dump); + seg_start = pmt_to_uint64(r); + extra_len = seg_start - METADATA_HEADER_SIZE; + } + + if(extra_len > 0) { + size = 0; + hdr_buffer = new char[extra_len]; + while(size < extra_len) { + ret = fread(&hdr_buffer[size], sizeof(char), extra_len-size, fp); + if(ret == 0) { + delete [] hdr_buffer; + if(feof(fp)) + return false; + else { + std::stringstream s; + s << "file_meta_source: error occurred extracting extras: " + << strerror(errno) << std::endl; + throw std::runtime_error(s.str()); + } + } + size += ret; + } + + str.clear(); + str.insert(0, hdr_buffer, extra_len); + extras = pmt_deserialize_str(str); + delete [] hdr_buffer; + } + + return true; + } + + void + file_meta_source_impl::parse_header(pmt_t hdr, uint64_t offset, + std::vector<gr_tag_t> &tags) + { + pmt_t r, key; + + // GET SAMPLE RATE + key = pmt_string_to_symbol("rx_rate"); + if(pmt_dict_has_key(hdr, key)) { + r = pmt_dict_ref(hdr, key, PMT_NIL); + d_samp_rate = pmt_to_double(r); + + gr_tag_t t; + t.offset = offset; + t.key = key; + t.value = r; + t.srcid = alias_pmt(); + tags.push_back(t); + } + else { + throw std::runtime_error("file_meta_source: Could not extract sample rate.\n"); + } + + // GET TIME STAMP + key = pmt_string_to_symbol("rx_time"); + if(pmt_dict_has_key(hdr, key)) { + d_time_stamp = pmt_dict_ref(hdr, key, PMT_NIL); + + gr_tag_t t; + t.offset = offset; + t.key = key; + t.value = d_time_stamp; + t.srcid = alias_pmt(); + tags.push_back(t); + } + else { + throw std::runtime_error("file_meta_source: Could not extract time stamp.\n"); + } + + // GET ITEM SIZE OF DATA + if(pmt_dict_has_key(hdr, pmt_string_to_symbol("size"))) { + d_itemsize = pmt_to_long(pmt_dict_ref(hdr, pmt_string_to_symbol("size"), PMT_NIL)); + } + else { + throw std::runtime_error("file_meta_source: Could not extract item size.\n"); + } + + // GET SEGMENT SIZE + if(pmt_dict_has_key(hdr, pmt_string_to_symbol("bytes"))) { + d_seg_size = pmt_to_uint64(pmt_dict_ref(hdr, pmt_string_to_symbol("bytes"), PMT_NIL)); + + // Convert from bytes to items + d_seg_size /= d_itemsize; + } + else { + throw std::runtime_error("file_meta_source: Could not extract segment size.\n"); + } + } + + void + file_meta_source_impl::parse_extras(pmt_t extras, uint64_t offset, + std::vector<gr_tag_t> &tags) + { + pmt_t item, key, val; + + size_t nitems = pmt_length(extras); + for(size_t i = 0; i < nitems; i++) { + item = pmt_nth(i, extras); + key = pmt_car(item); + val = pmt_cdr(item); + + gr_tag_t t; + t.offset = offset; + t.key = key; + t.value = val; + t.srcid = alias_pmt(); + tags.push_back(t); + } + } + + bool + file_meta_source_impl::open(const std::string &filename, + const std::string &hdr_filename) + { + bool ret = true; + if(d_state == STATE_DETACHED) { + std::string s; + if(hdr_filename == "") + s = filename + ".hdr"; + else + s = hdr_filename; + ret = _open(&d_new_hdr_fp, s.c_str()); + } + + ret = ret && _open(&d_new_fp, filename.c_str()); + d_updated = true; + return ret; + } + + bool + file_meta_source_impl::_open(FILE **fp, const char *filename) + { + gruel::scoped_lock guard(d_mutex); // hold mutex for duration of this function + + bool ret = true; + int fd; + + if((fd = ::open(filename, + O_RDONLY|OUR_O_LARGEFILE|OUR_O_BINARY)) < 0) { + perror(filename); + return false; + } + + if(*fp) { // if we've already got a new one open, close it + fclose(*fp); + fp = 0; + } + + if((*fp = fdopen(fd, "rb")) == NULL) { + perror(filename); + ::close(fd); // don't leak file descriptor if fdopen fails. + } + + ret = fp != 0; + + return ret; + } + + void + file_meta_source_impl::close() + { + gruel::scoped_lock guard(d_mutex); // hold mutex for duration of this function + if(d_state == STATE_DETACHED) { + if(d_new_hdr_fp) { + fclose(d_new_hdr_fp); + d_new_hdr_fp = 0; + } + } + + if(d_new_fp) { + fclose(d_new_fp); + d_new_fp = 0; + } + d_updated = true; + } + + void + file_meta_source_impl::do_update() + { + if(d_updated) { + gruel::scoped_lock guard(d_mutex); // hold mutex for duration of this block + if(d_state == STATE_DETACHED) { + if(d_hdr_fp) + fclose(d_hdr_fp); + d_hdr_fp = d_new_hdr_fp; // install new file pointer + d_new_hdr_fp = 0; + } + + if(d_fp) + fclose(d_fp); + d_fp = d_new_fp; // install new file pointer + d_new_fp = 0; + + d_updated = false; + } + } + + int + file_meta_source_impl::work(int noutput_items, + gr_vector_const_void_star &input_items, + gr_vector_void_star &output_items) + { + // We've reached the end of a segment; parse the next header and get + // the new tags to send and set the next segment size. + if(d_seg_size == 0) { + pmt_t hdr=PMT_NIL, extras=PMT_NIL; + if(read_header(hdr, extras)) { + parse_header(hdr, nitems_written(0), d_tags); + parse_extras(extras, nitems_written(0), d_tags); + } + else { + return -1; + } + } + + char *out = (char*)output_items[0]; + int i; + int seg_size = std::min(noutput_items, (int)d_seg_size); + int size = seg_size; + + do_update(); // update d_fp is reqd + if(d_fp == NULL) + throw std::runtime_error("work with file not open"); + + // Push all tags onto the stream and remove them from the vector + while(!d_tags.empty()) { + add_item_tag(0, d_tags.back()); + d_tags.pop_back(); + } + + gruel::scoped_lock lock(d_mutex); // hold for the rest of this function + while(size) { + i = fread(out, d_itemsize, size, d_fp); + + size -= i; + d_seg_size -= i; + out += i * d_itemsize; + + if(size == 0) // done + break; + + if(i > 0) // short read, try again + continue; + + // We got a zero from fread. This is either EOF or error. In + // any event, if we're in repeat mode, seek back to the beginning + // of the file and try again, else break + + if(!d_repeat) + break; + + if(fseek(d_fp, 0, SEEK_SET) == -1) { + std::stringstream s; + s << "[" << __FILE__ << "]" << " fseek failed" << std::endl; + throw std::runtime_error(s.str()); + } + } + + if(size > 0) { // EOF or error + if(size == seg_size) // we didn't read anything; say we're done + return -1; + return seg_size - size; // else return partial result + } + + return seg_size; + } + + } /* namespace blocks */ +} /* namespace gr */ diff --git a/gr-blocks/lib/file_meta_source_impl.h b/gr-blocks/lib/file_meta_source_impl.h new file mode 100644 index 0000000000..ca7ddc6e10 --- /dev/null +++ b/gr-blocks/lib/file_meta_source_impl.h @@ -0,0 +1,89 @@ +/* -*- c++ -*- */ +/* + * Copyright 2012 Free Software Foundation, Inc. + * + * This file is part of GNU Radio + * + * GNU Radio is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 3, or (at your option) + * any later version. + * + * GNU Radio is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with GNU Radio; see the file COPYING. If not, write to + * the Free Software Foundation, Inc., 51 Franklin Street, + * Boston, MA 02110-1301, USA. + */ + +#ifndef INCLUDED_BLOCKS_FILE_META_SOURCE_IMPL_H +#define INCLUDED_BLOCKS_FILE_META_SOURCE_IMPL_H + +#include <blocks/file_meta_source.h> +#include <gr_tags.h> +#include <gruel/pmt.h> +#include <gruel/thread.h> + +#include <blocks/file_meta_sink.h> + +using namespace pmt; + +namespace gr { + namespace blocks { + + class file_meta_source_impl : public file_meta_source + { + private: + enum meta_state_t { + STATE_INLINE=0, + STATE_DETACHED + }; + + size_t d_itemsize; + double d_samp_rate; + pmt_t d_time_stamp; + size_t d_seg_size; + bool d_updated; + bool d_repeat; + + gruel::mutex d_mutex; + FILE *d_new_fp, *d_new_hdr_fp; + FILE *d_fp, *d_hdr_fp; + meta_state_t d_state; + + std::vector<gr_tag_t> d_tags; + + protected: + bool _open(FILE **fp, const char *filename); + bool read_header(pmt_t &hdr, pmt_t &extras); + void parse_header(pmt_t hdr, uint64_t offset, + std::vector<gr_tag_t> &tags); + void parse_extras(pmt_t extras, uint64_t offset, + std::vector<gr_tag_t> &tags); + + public: + file_meta_source_impl(const std::string &filename, + bool repeat=false, + bool detached_header=false, + const std::string &hdr_filename=""); + + ~file_meta_source_impl(); + + bool open(const std::string &filename, + const std::string &hdr_filename=""); + void close(); + void do_update(); + + int work(int noutput_items, + gr_vector_const_void_star &input_items, + gr_vector_void_star &output_items); + }; + + } /* namespace blocks */ +} /* namespace gr */ + +#endif /* INCLUDED_BLOCKS_FILE_META_SOURCE_IMPL_H */ diff --git a/gr-blocks/python/CMakeLists.txt b/gr-blocks/python/CMakeLists.txt index 710ab155ce..cab0b956f7 100644 --- a/gr-blocks/python/CMakeLists.txt +++ b/gr-blocks/python/CMakeLists.txt @@ -23,6 +23,7 @@ include(GrPython) GR_PYTHON_INSTALL( FILES __init__.py + parse_file_metadata.py DESTINATION ${GR_PYTHON_DIR}/gnuradio/blocks COMPONENT "blocks_python" ) diff --git a/gr-blocks/python/parse_file_metadata.py b/gr-blocks/python/parse_file_metadata.py new file mode 100644 index 0000000000..ec7bf6e804 --- /dev/null +++ b/gr-blocks/python/parse_file_metadata.py @@ -0,0 +1,188 @@ +#!/usr/bin/env python +# +# Copyright 2012 Free Software Foundation, Inc. +# +# This file is part of GNU Radio +# +# GNU Radio is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3, or (at your option) +# any later version. +# +# GNU Radio is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with GNU Radio; see the file COPYING. If not, write to +# the Free Software Foundation, Inc., 51 Franklin Street, +# Boston, MA 02110-1301, USA. +# + +import sys +from gnuradio import gr +from gruel import pmt + +try: + import blocks_swig as blocks +except: + from gnuradio import blocks + +''' +sr Sample rate (samples/second) +time Time as uint64(secs), double(fractional secs) +type Type of data (see gr_file_types enum) +cplx is complex? (True or False) +strt Start of data (or size of header) in bytes +size Size of data in bytes +''' + +HEADER_LENGTH = blocks.METADATA_HEADER_SIZE + +ftype_to_string = {blocks.GR_FILE_BYTE: "bytes", + blocks.GR_FILE_SHORT: "short", + blocks.GR_FILE_INT: "int", + blocks.GR_FILE_LONG: "long", + blocks.GR_FILE_LONG_LONG: "long long", + blocks.GR_FILE_FLOAT: "float", + blocks.GR_FILE_DOUBLE: "double" } + +ftype_to_size = {blocks.GR_FILE_BYTE: gr.sizeof_char, + blocks.GR_FILE_SHORT: gr.sizeof_short, + blocks.GR_FILE_INT: gr.sizeof_int, + blocks.GR_FILE_LONG: gr.sizeof_int, + blocks.GR_FILE_LONG_LONG: 2*gr.sizeof_int, + blocks.GR_FILE_FLOAT: gr.sizeof_float, + blocks.GR_FILE_DOUBLE: gr.sizeof_double} + +def parse_header(p, VERBOSE=False): + dump = pmt.PMT_NIL + + info = dict() + + if(pmt.pmt_is_dict(p) is False): + sys.stderr.write("Header is not a PMT dictionary: invalid or corrupt data file.\n") + sys.exit(1) + + # GET FILE FORMAT VERSION NUMBER + if(pmt.pmt_dict_has_key(p, pmt.pmt_string_to_symbol("version"))): + r = pmt.pmt_dict_ref(p, pmt.pmt_string_to_symbol("version"), dump) + version = pmt.pmt_to_long(r) + if(VERBOSE): + print "Version Number: {0}".format(version) + else: + sys.stderr.write("Could not find key 'sr': invalid or corrupt data file.\n") + sys.exit(1) + + # EXTRACT SAMPLE RATE + if(pmt.pmt_dict_has_key(p, pmt.pmt_string_to_symbol("rx_rate"))): + r = pmt.pmt_dict_ref(p, pmt.pmt_string_to_symbol("rx_rate"), dump) + samp_rate = pmt.pmt_to_double(r) + info["rx_rate"] = samp_rate + if(VERBOSE): + print "Sample Rate: {0:.2f} sps".format(samp_rate) + else: + sys.stderr.write("Could not find key 'sr': invalid or corrupt data file.\n") + sys.exit(1) + + # EXTRACT TIME STAMP + if(pmt.pmt_dict_has_key(p, pmt.pmt_string_to_symbol("rx_time"))): + r = pmt.pmt_dict_ref(p, pmt.pmt_string_to_symbol("rx_time"), dump) + pmt_secs = pmt.pmt_tuple_ref(r, 0) + pmt_fracs = pmt.pmt_tuple_ref(r, 1) + secs = float(pmt.pmt_to_uint64(pmt_secs)) + fracs = pmt.pmt_to_double(pmt_fracs) + t = secs + fracs + info["rx_time"] = t + if(VERBOSE): + print "Seconds: {0:.6f}".format(t) + else: + sys.stderr.write("Could not find key 'time': invalid or corrupt data file.\n") + sys.exit(1) + + # EXTRACT ITEM SIZE + if(pmt.pmt_dict_has_key(p, pmt.pmt_string_to_symbol("size"))): + r = pmt.pmt_dict_ref(p, pmt.pmt_string_to_symbol("size"), dump) + dsize = pmt.pmt_to_long(r) + info["size"] = dsize + if(VERBOSE): + print "Item size: {0}".format(dsize) + else: + sys.stderr.write("Could not find key 'size': invalid or corrupt data file.\n") + sys.exit(1) + + # EXTRACT DATA TYPE + if(pmt.pmt_dict_has_key(p, pmt.pmt_string_to_symbol("type"))): + r = pmt.pmt_dict_ref(p, pmt.pmt_string_to_symbol("type"), dump) + dtype = pmt.pmt_to_long(r) + stype = ftype_to_string[dtype] + info["type"] = stype + if(VERBOSE): + print "Data Type: {0} ({1})".format(stype, dtype) + else: + sys.stderr.write("Could not find key 'type': invalid or corrupt data file.\n") + sys.exit(1) + + # EXTRACT COMPLEX + if(pmt.pmt_dict_has_key(p, pmt.pmt_string_to_symbol("cplx"))): + r = pmt.pmt_dict_ref(p, pmt.pmt_string_to_symbol("cplx"), dump) + cplx = pmt.pmt_to_bool(r) + info["cplx"] = cplx + if(VERBOSE): + print "Complex? {0}".format(cplx) + else: + sys.stderr.write("Could not find key 'cplx': invalid or corrupt data file.\n") + sys.exit(1) + + # EXTRACT WHERE CURRENT SEGMENT STARTS + if(pmt.pmt_dict_has_key(p, pmt.pmt_string_to_symbol("strt"))): + r = pmt.pmt_dict_ref(p, pmt.pmt_string_to_symbol("strt"), dump) + seg_start = pmt.pmt_to_uint64(r) + info["hdr_len"] = seg_start + info["extra_len"] = seg_start - HEADER_LENGTH + info["has_extra"] = info["extra_len"] > 0 + if(VERBOSE): + print "Header Length: {0} bytes".format(info["hdr_len"]) + print "Extra Length: {0}".format((info["extra_len"])) + print "Extra Header? {0}".format(info["has_extra"]) + else: + sys.stderr.write("Could not find key 'strt': invalid or corrupt data file.\n") + sys.exit(1) + + # EXTRACT SIZE OF DATA + if(pmt.pmt_dict_has_key(p, pmt.pmt_string_to_symbol("bytes"))): + r = pmt.pmt_dict_ref(p, pmt.pmt_string_to_symbol("bytes"), dump) + nbytes = pmt.pmt_to_uint64(r) + + nitems = nbytes/dsize + info["nitems"] = nitems + info["nbytes"] = nbytes + + if(VERBOSE): + print "Size of Data: {0} bytes".format(nbytes) + print " {0} items".format(nitems) + else: + sys.stderr.write("Could not find key 'size': invalid or corrupt data file.\n") + sys.exit(1) + + return info + +# IF THERE IS EXTRA DATA, PULL OUT THE DICTIONARY AND PARSE IT +def parse_extra_dict(p, info, VERBOSE=False): + if(pmt.pmt_is_dict(p) is False): + sys.stderr.write("Extra header is not a PMT dictionary: invalid or corrupt data file.\n") + sys.exit(1) + + items = pmt.pmt_dict_items(p) + nitems = pmt.pmt_length(items) + for i in xrange(nitems): + item = pmt.pmt_nth(i, items) + key = pmt.pmt_symbol_to_string(pmt.pmt_car(item)) + val = pmt.pmt_cdr(item) + info[key] = val + if(VERBOSE): + print "{0}: ".format(key) + pmt.pmt_print(val) + + return info diff --git a/gr-blocks/python/qa_file_metadata.py b/gr-blocks/python/qa_file_metadata.py new file mode 100644 index 0000000000..9f4a331d65 --- /dev/null +++ b/gr-blocks/python/qa_file_metadata.py @@ -0,0 +1,197 @@ +#!/usr/bin/env python +# +# Copyright 2012 Free Software Foundation, Inc. +# +# This file is part of GNU Radio +# +# GNU Radio is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3, or (at your option) +# any later version. +# +# GNU Radio is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with GNU Radio; see the file COPYING. If not, write to +# the Free Software Foundation, Inc., 51 Franklin Street, +# Boston, MA 02110-1301, USA. +# + +from gnuradio import gr, gr_unittest +import parse_file_metadata +import blocks_swig as blocks +import pmt +import os, time + +class test_file_metadata(gr_unittest.TestCase): + + def setUp(self): + self.tb = gr.top_block() + + def tearDown(self): + self.tb = None + + def test_001(self): + outfile = "test_out.dat" + + detached = False + samp_rate = 200000 + key = pmt.pmt_intern("samp_rate") + val = pmt.pmt_from_double(samp_rate) + extras = pmt.pmt_make_dict() + extras = pmt.pmt_dict_add(extras, key, val) + extras_str = pmt.pmt_serialize_str(extras) + + src = gr.sig_source_c(samp_rate, gr.GR_COS_WAVE, 1000, 1, 0) + head = gr.head(gr.sizeof_gr_complex, 1000) + fsnk = blocks.file_meta_sink(gr.sizeof_gr_complex, outfile, + samp_rate, 1, + blocks.GR_FILE_FLOAT, True, + 1000000, extras_str, detached) + fsnk.set_unbuffered(True) + + self.tb.connect(src, head, fsnk) + self.tb.run() + fsnk.close() + + handle = open(outfile, "rb") + header_str = handle.read(parse_file_metadata.HEADER_LENGTH) + if(len(header_str) == 0): + self.assertFalse() + + try: + header = pmt.pmt_deserialize_str(header_str) + except RuntimeError: + self.assertFalse() + + info = parse_file_metadata.parse_header(header, False) + + extra_str = handle.read(info["extra_len"]) + self.assertGreater(len(extra_str), 0) + handle.close() + + try: + extra = pmt.pmt_deserialize_str(extra_str) + except RuntimeError: + self.assertFalse() + + extra_info = parse_file_metadata.parse_extra_dict(extra, info, False) + + self.assertEqual(info['rx_rate'], samp_rate) + self.assertEqual(pmt.pmt_to_double(extra_info['samp_rate']), samp_rate) + + + # Test file metadata source + # Create a new sig source to start from the beginning + src2 = gr.sig_source_c(samp_rate, gr.GR_COS_WAVE, 1000, 1, 0) + fsrc = blocks.file_meta_source(outfile, False) + vsnk = gr.vector_sink_c() + tsnk = gr.tag_debug(gr.sizeof_gr_complex, "QA") + ssnk = gr.vector_sink_c() + head.reset() + self.tb.disconnect(src, head, fsnk) + self.tb.connect(fsrc, vsnk) + self.tb.connect(fsrc, tsnk) + self.tb.connect(src2, head, ssnk) + self.tb.run() + + # Test to make sure tags with 'samp_rate' and 'rx_rate' keys + # were generated and received correctly. + tags = tsnk.current_tags() + for t in tags: + if(pmt.pmt_eq(t.key, pmt.pmt_intern("samp_rate"))): + self.assertEqual(pmt.pmt_to_double(t.value), samp_rate) + elif(pmt.pmt_eq(t.key, pmt.pmt_intern("rx_rate"))): + self.assertEqual(pmt.pmt_to_double(t.value), samp_rate) + + # Test that the data portion was extracted and received correctly. + self.assertComplexTuplesAlmostEqual(vsnk.data(), ssnk.data(), 5) + + os.remove(outfile) + + def test_002(self): + outfile = "test_out.dat" + outfile_hdr = "test_out.dat.hdr" + + detached = True + samp_rate = 200000 + key = pmt.pmt_intern("samp_rate") + val = pmt.pmt_from_double(samp_rate) + extras = pmt.pmt_make_dict() + extras = pmt.pmt_dict_add(extras, key, val) + extras_str = pmt.pmt_serialize_str(extras) + + src = gr.sig_source_c(samp_rate, gr.GR_COS_WAVE, 1000, 1, 0) + head = gr.head(gr.sizeof_gr_complex, 1000) + fsnk = blocks.file_meta_sink(gr.sizeof_gr_complex, outfile, + samp_rate, 1, + blocks.GR_FILE_FLOAT, True, + 1000000, extras_str, detached) + fsnk.set_unbuffered(True) + + self.tb.connect(src, head, fsnk) + self.tb.run() + fsnk.close() + + # Open detached header for reading + handle = open(outfile_hdr, "rb") + header_str = handle.read(parse_file_metadata.HEADER_LENGTH) + if(len(header_str) == 0): + self.assertFalse() + + try: + header = pmt.pmt_deserialize_str(header_str) + except RuntimeError: + self.assertFalse() + + info = parse_file_metadata.parse_header(header, False) + + extra_str = handle.read(info["extra_len"]) + self.assertGreater(len(extra_str), 0) + handle.close() + + try: + extra = pmt.pmt_deserialize_str(extra_str) + except RuntimeError: + self.assertFalse() + + extra_info = parse_file_metadata.parse_extra_dict(extra, info, False) + + self.assertEqual(info['rx_rate'], samp_rate) + self.assertEqual(pmt.pmt_to_double(extra_info['samp_rate']), samp_rate) + + + # Test file metadata source + # Create a new sig source to start from the beginning + src2 = gr.sig_source_c(samp_rate, gr.GR_COS_WAVE, 1000, 1, 0) + fsrc = blocks.file_meta_source(outfile, False, detached, outfile_hdr) + vsnk = gr.vector_sink_c() + tsnk = gr.tag_debug(gr.sizeof_gr_complex, "QA") + ssnk = gr.vector_sink_c() + head.reset() + self.tb.disconnect(src, head, fsnk) + self.tb.connect(fsrc, vsnk) + self.tb.connect(fsrc, tsnk) + self.tb.connect(src2, head, ssnk) + self.tb.run() + + # Test to make sure tags with 'samp_rate' and 'rx_rate' keys + # were generated and received correctly. + tags = tsnk.current_tags() + for t in tags: + if(pmt.pmt_eq(t.key, pmt.pmt_intern("samp_rate"))): + self.assertEqual(pmt.pmt_to_double(t.value), samp_rate) + elif(pmt.pmt_eq(t.key, pmt.pmt_intern("rx_rate"))): + self.assertEqual(pmt.pmt_to_double(t.value), samp_rate) + + # Test that the data portion was extracted and received correctly. + self.assertComplexTuplesAlmostEqual(vsnk.data(), ssnk.data(), 5) + + os.remove(outfile) + os.remove(outfile_hdr) + +if __name__ == '__main__': + gr_unittest.run(test_file_metadata, "test_file_metadata.xml") diff --git a/gr-blocks/swig/blocks_swig.i b/gr-blocks/swig/blocks_swig.i index 7ec6bb423f..53eea818fb 100644 --- a/gr-blocks/swig/blocks_swig.i +++ b/gr-blocks/swig/blocks_swig.i @@ -62,6 +62,8 @@ #include "blocks/divide_ii.h" #include "blocks/divide_cc.h" #include "blocks/file_source.h" +#include "blocks/file_meta_sink.h" +#include "blocks/file_meta_source.h" #include "blocks/float_to_char.h" #include "blocks/float_to_complex.h" #include "blocks/float_to_int.h" @@ -147,6 +149,8 @@ %include "blocks/conjugate_cc.h" %include "blocks/deinterleave.h" %include "blocks/file_source.h" +%include "blocks/file_meta_sink.h" +%include "blocks/file_meta_source.h" %include "blocks/divide_ff.h" %include "blocks/divide_ss.h" %include "blocks/divide_ii.h" @@ -239,6 +243,8 @@ GR_SWIG_BLOCK_MAGIC2(blocks, divide_ss); GR_SWIG_BLOCK_MAGIC2(blocks, divide_ii); GR_SWIG_BLOCK_MAGIC2(blocks, divide_cc); GR_SWIG_BLOCK_MAGIC2(blocks, file_source); +GR_SWIG_BLOCK_MAGIC2(blocks, file_meta_sink); +GR_SWIG_BLOCK_MAGIC2(blocks, file_meta_source); GR_SWIG_BLOCK_MAGIC2(blocks, float_to_char); GR_SWIG_BLOCK_MAGIC2(blocks, float_to_complex); GR_SWIG_BLOCK_MAGIC2(blocks, float_to_int); diff --git a/gr-utils/src/python/CMakeLists.txt b/gr-utils/src/python/CMakeLists.txt index 0544d815f2..4b956904f3 100644 --- a/gr-utils/src/python/CMakeLists.txt +++ b/gr-utils/src/python/CMakeLists.txt @@ -49,6 +49,7 @@ GR_PYTHON_INSTALL( gr_plot_iq gr_plot_short gr_plot_qt + gr_read_file_metadata grcc DESTINATION ${GR_RUNTIME_DIR} COMPONENT "utils" diff --git a/gr-utils/src/python/gr_read_file_metadata b/gr-utils/src/python/gr_read_file_metadata new file mode 100644 index 0000000000..cf0cd5b116 --- /dev/null +++ b/gr-utils/src/python/gr_read_file_metadata @@ -0,0 +1,87 @@ +#!/usr/bin/env python +# +# Copyright 2012 Free Software Foundation, Inc. +# +# This file is part of GNU Radio +# +# GNU Radio is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3, or (at your option) +# any later version. +# +# GNU Radio is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with GNU Radio; see the file COPYING. If not, write to +# the Free Software Foundation, Inc., 51 Franklin Street, +# Boston, MA 02110-1301, USA. +# + +import sys +from optparse import OptionParser + +from gruel import pmt +from gnuradio.blocks import parse_file_metadata + +def main(filename, detached=False): + handle = open(filename, "rb") + + nheaders = 0 + nread = 0 + while(True): + # read out next header bytes + hdr_start = handle.tell() + header_str = handle.read(parse_file_metadata.HEADER_LENGTH) + if(len(header_str) == 0): + break + + # Convert from string to PMT (should be a dictionary) + try: + header = pmt.pmt_deserialize_str(header_str) + except RuntimeError: + sys.stderr.write("Could not deserialize header: invalid or corrupt data file.\n") + sys.exit(1) + + print "HEADER {0}".format(nheaders) + info = parse_file_metadata.parse_header(header, True) + + if(info["extra_len"] > 0): + extra_str = handle.read(info["extra_len"]) + if(len(extra_str) == 0): + break + + try: + extra = pmt.pmt_deserialize_str(extra_str) + except RuntimeError: + sys.stderr.write("Could not deserialize extras: invalid or corrupt data file.\n") + sys.exit(1) + + print "\nExtra Header:" + extra_info = parse_file_metadata.parse_extra_dict(extra, info, True) + + nheaders += 1 + nread += parse_file_metadata.HEADER_LENGTH + info["extra_len"] + if(not detached): + nread += info['nbytes'] + handle.seek(nread, 0) + print "\n\n" + +if __name__ == "__main__": + usage="%prog: [options] filename" + description = "Read in a GNU Radio file with meta data, extracts the header and prints it." + + parser = OptionParser(conflict_handler="resolve", + usage=usage, description=description) + parser.add_option("-D", "--detached", action="store_true", default=False, + help="Used if header is detached.") + (options, args) = parser.parse_args () + + if(len(args) < 1): + sys.stderr.write("No filename given\n") + sys.exit(1) + + filename = args[0] + main(filename, options.detached) diff --git a/gruel/src/include/gruel/pmt.h b/gruel/src/include/gruel/pmt.h index d09686783c..68b122105a 100644 --- a/gruel/src/include/gruel/pmt.h +++ b/gruel/src/include/gruel/pmt.h @@ -32,6 +32,7 @@ #include <stdint.h> #include <iosfwd> #include <stdexcept> +#include <vector> namespace gruel { class msg_accepter; @@ -412,17 +413,29 @@ GRUEL_API pmt_t pmt_make_c32vector(size_t k, std::complex<float> fill); GRUEL_API pmt_t pmt_make_c64vector(size_t k, std::complex<double> fill); GRUEL_API pmt_t pmt_init_u8vector(size_t k, const uint8_t *data); +GRUEL_API pmt_t pmt_init_u8vector(size_t k, const std::vector<uint8_t> &data); GRUEL_API pmt_t pmt_init_s8vector(size_t k, const int8_t *data); +GRUEL_API pmt_t pmt_init_s8vector(size_t k, const std::vector<int8_t> &data); GRUEL_API pmt_t pmt_init_u16vector(size_t k, const uint16_t *data); +GRUEL_API pmt_t pmt_init_u16vector(size_t k, const std::vector<uint16_t> &data); GRUEL_API pmt_t pmt_init_s16vector(size_t k, const int16_t *data); +GRUEL_API pmt_t pmt_init_s16vector(size_t k, const std::vector<int16_t> &data); GRUEL_API pmt_t pmt_init_u32vector(size_t k, const uint32_t *data); +GRUEL_API pmt_t pmt_init_u32vector(size_t k, const std::vector<uint32_t> &data); GRUEL_API pmt_t pmt_init_s32vector(size_t k, const int32_t *data); +GRUEL_API pmt_t pmt_init_s32vector(size_t k, const std::vector<int32_t> &data); GRUEL_API pmt_t pmt_init_u64vector(size_t k, const uint64_t *data); +GRUEL_API pmt_t pmt_init_u64vector(size_t k, const std::vector<uint64_t> &data); GRUEL_API pmt_t pmt_init_s64vector(size_t k, const int64_t *data); +GRUEL_API pmt_t pmt_init_s64vector(size_t k, const std::vector<int64_t> &data); GRUEL_API pmt_t pmt_init_f32vector(size_t k, const float *data); +GRUEL_API pmt_t pmt_init_f32vector(size_t k, const std::vector<float> &data); GRUEL_API pmt_t pmt_init_f64vector(size_t k, const double *data); +GRUEL_API pmt_t pmt_init_f64vector(size_t k, const std::vector<double> &data); GRUEL_API pmt_t pmt_init_c32vector(size_t k, const std::complex<float> *data); +GRUEL_API pmt_t pmt_init_c32vector(size_t k, const std::vector<std::complex<float> > &data); GRUEL_API pmt_t pmt_init_c64vector(size_t k, const std::complex<double> *data); +GRUEL_API pmt_t pmt_init_c64vector(size_t k, const std::vector<std::complex<double> > &data); GRUEL_API uint8_t pmt_u8vector_ref(pmt_t v, size_t k); GRUEL_API int8_t pmt_s8vector_ref(pmt_t v, size_t k); diff --git a/gruel/src/lib/pmt/pmt_serialize.cc b/gruel/src/lib/pmt/pmt_serialize.cc index a19809a667..1ee7ca1fe7 100644 --- a/gruel/src/lib/pmt/pmt_serialize.cc +++ b/gruel/src/lib/pmt/pmt_serialize.cc @@ -309,11 +309,192 @@ pmt_serialize(pmt_t obj, std::streambuf &sb) } - if (pmt_is_vector(obj)) - throw pmt_notimplemented("pmt_serialize (vector)", obj); + if (pmt_is_vector(obj)) { + size_t vec_len = pmt::pmt_length(obj); + ok = serialize_untagged_u8(PST_VECTOR, sb); + ok &= serialize_untagged_u32(vec_len, sb); + for(size_t i=0; i<vec_len; i++) { + ok &= pmt_serialize(pmt_vector_ref(obj, i), sb); + } + return ok; + } + + if (pmt_is_uniform_vector(obj)) { + size_t npad = 1; + size_t vec_len = pmt::pmt_length(obj); + + if(pmt_is_u8vector(obj)) { + ok = serialize_untagged_u8(PST_UNIFORM_VECTOR, sb); + ok &= serialize_untagged_u8(UVI_U8, sb); + ok &= serialize_untagged_u32(vec_len, sb); + ok &= serialize_untagged_u8(npad, sb); + for(size_t i=0; i<npad; i++) { + ok &= serialize_untagged_u8(0, sb); + } + for(size_t i=0; i<vec_len; i++) { + ok &= serialize_untagged_u8(pmt_u8vector_ref(obj, i), sb); + } + return ok; + } + + if(pmt_is_s8vector(obj)) { + ok = serialize_untagged_u8(PST_UNIFORM_VECTOR, sb); + ok &= serialize_untagged_u8(UVI_S8, sb); + ok &= serialize_untagged_u32(vec_len, sb); + ok &= serialize_untagged_u8(npad, sb); + for(size_t i=0; i<npad; i++) { + ok &= serialize_untagged_u8(0, sb); + } + for(size_t i=0; i<vec_len; i++) { + ok &= serialize_untagged_u8(pmt_s8vector_ref(obj, i), sb); + } + return ok; + } + + if(pmt_is_u16vector(obj)) { + ok = serialize_untagged_u8(PST_UNIFORM_VECTOR, sb); + ok &= serialize_untagged_u8(UVI_U16, sb); + ok &= serialize_untagged_u32(vec_len, sb); + ok &= serialize_untagged_u8(npad, sb); + for(size_t i=0; i<npad; i++) { + ok &= serialize_untagged_u8(0, sb); + } + for(size_t i=0; i<vec_len; i++) { + ok &= serialize_untagged_u16(pmt_u16vector_ref(obj, i), sb); + } + return ok; + } + + if(pmt_is_s16vector(obj)) { + ok = serialize_untagged_u8(PST_UNIFORM_VECTOR, sb); + ok &= serialize_untagged_u8(UVI_S16, sb); + ok &= serialize_untagged_u32(vec_len, sb); + ok &= serialize_untagged_u8(npad, sb); + for(size_t i=0; i<npad; i++) { + ok &= serialize_untagged_u8(0, sb); + } + for(size_t i=0; i<vec_len; i++) { + ok &= serialize_untagged_u16(pmt_s16vector_ref(obj, i), sb); + } + return ok; + } - if (pmt_is_uniform_vector(obj)) - throw pmt_notimplemented("pmt_serialize (uniform-vector)", obj); + if(pmt_is_u32vector(obj)) { + ok = serialize_untagged_u8(PST_UNIFORM_VECTOR, sb); + ok &= serialize_untagged_u8(UVI_U32, sb); + ok &= serialize_untagged_u32(vec_len, sb); + ok &= serialize_untagged_u8(npad, sb); + for(size_t i=0; i<npad; i++) { + ok &= serialize_untagged_u8(0, sb); + } + for(size_t i=0; i<vec_len; i++) { + ok &= serialize_untagged_u32(pmt_u32vector_ref(obj, i), sb); + } + return ok; + } + + if(pmt_is_s32vector(obj)) { + ok = serialize_untagged_u8(PST_UNIFORM_VECTOR, sb); + ok &= serialize_untagged_u8(UVI_S32, sb); + ok &= serialize_untagged_u32(vec_len, sb); + ok &= serialize_untagged_u8(npad, sb); + for(size_t i=0; i<npad; i++) { + ok &= serialize_untagged_u8(0, sb); + } + for(size_t i=0; i<vec_len; i++) { + ok &= serialize_untagged_u32(pmt_s32vector_ref(obj, i), sb); + } + return ok; + } + + if(pmt_is_u64vector(obj)) { + ok = serialize_untagged_u8(PST_UNIFORM_VECTOR, sb); + ok &= serialize_untagged_u8(UVI_U64, sb); + ok &= serialize_untagged_u32(vec_len, sb); + ok &= serialize_untagged_u8(npad, sb); + for(size_t i=0; i<npad; i++) { + ok &= serialize_untagged_u8(0, sb); + } + for(size_t i=0; i<vec_len; i++) { + ok &= serialize_untagged_u64(pmt_u64vector_ref(obj, i), sb); + } + return ok; + } + + if(pmt_is_s64vector(obj)) { + ok = serialize_untagged_u8(PST_UNIFORM_VECTOR, sb); + ok &= serialize_untagged_u8(UVI_S64, sb); + ok &= serialize_untagged_u32(vec_len, sb); + ok &= serialize_untagged_u8(npad, sb); + for(size_t i=0; i<npad; i++) { + ok &= serialize_untagged_u8(0, sb); + } + for(size_t i=0; i<vec_len; i++) { + ok &= serialize_untagged_u64(pmt_s64vector_ref(obj, i), sb); + } + return ok; + } + + if(pmt_is_f32vector(obj)) { + ok = serialize_untagged_u8(PST_UNIFORM_VECTOR, sb); + ok &= serialize_untagged_u8(UVI_F32, sb); + ok &= serialize_untagged_u32(vec_len, sb); + ok &= serialize_untagged_u8(npad, sb); + for(size_t i=0; i<npad; i++) { + ok &= serialize_untagged_u8(0, sb); + } + for(size_t i=0; i<vec_len; i++) { + ok &= serialize_untagged_f64(pmt_f32vector_ref(obj, i), sb); + } + return ok; + } + + if(pmt_is_f64vector(obj)) { + ok = serialize_untagged_u8(PST_UNIFORM_VECTOR, sb); + ok &= serialize_untagged_u8(UVI_F64, sb); + ok &= serialize_untagged_u32(vec_len, sb); + ok &= serialize_untagged_u8(npad, sb); + for(size_t i=0; i<npad; i++) { + ok &= serialize_untagged_u8(0, sb); + } + for(size_t i=0; i<vec_len; i++) { + ok &= serialize_untagged_f64(pmt_f64vector_ref(obj, i), sb); + } + return ok; + } + + if(pmt_is_c32vector(obj)) { + ok = serialize_untagged_u8(PST_UNIFORM_VECTOR, sb); + ok &= serialize_untagged_u8(UVI_C32, sb); + ok &= serialize_untagged_u32(vec_len, sb); + ok &= serialize_untagged_u8(npad, sb); + for(size_t i=0; i<npad; i++) { + ok &= serialize_untagged_u8(0, sb); + } + for(size_t i=0; i<vec_len; i++) { + std::complex<float> c = pmt_c32vector_ref(obj, i); + ok &= serialize_untagged_f64(c.real(), sb); + ok &= serialize_untagged_f64(c.imag(), sb); + } + return ok; + } + + if(pmt_is_c64vector(obj)) { + ok = serialize_untagged_u8(PST_UNIFORM_VECTOR, sb); + ok &= serialize_untagged_u8(UVI_C64, sb); + ok &= serialize_untagged_u32(vec_len, sb); + ok &= serialize_untagged_u8(npad, sb); + for(size_t i=0; i<npad; i++) { + ok &= serialize_untagged_u8(0, sb); + } + for(size_t i=0; i<vec_len; i++) { + std::complex<double> c = pmt_c64vector_ref(obj, i); + ok &= serialize_untagged_f64(c.real(), sb); + ok &= serialize_untagged_f64(c.imag(), sb); + } + return ok; + } + } if (pmt_is_dict(obj)) throw pmt_notimplemented("pmt_serialize (dict)", obj); @@ -342,7 +523,7 @@ pmt_t pmt_deserialize(std::streambuf &sb) { uint8_t tag; - //uint8_t u8; + uint8_t u8; uint16_t u16; uint32_t u32; uint64_t u64; @@ -408,8 +589,159 @@ pmt_deserialize(std::streambuf &sb) } case PST_VECTOR: - case PST_DICT: + { + uint32_t nitems; + if(!deserialize_untagged_u32(&nitems, sb)) + goto error; + pmt_t vec = pmt_make_vector(nitems, PMT_NIL); + for(uint32_t i=0; i<nitems; i++) { + pmt_t item = pmt_deserialize(sb); + pmt_vector_set(vec, i, item); + } + return vec; + } + case PST_UNIFORM_VECTOR: + { + uint8_t utag, npad; + uint32_t nitems; + + if(!deserialize_untagged_u8(&utag, sb)) + return PMT_EOF; + + if(!deserialize_untagged_u32(&nitems, sb)) + goto error; + + deserialize_untagged_u8(&npad, sb); + for(size_t i; i < npad; i++) + deserialize_untagged_u8(&u8, sb); + + switch(utag) { + case(UVI_U8): + { + pmt_t vec = pmt_make_u8vector(nitems, 0); + for(uint32_t i=0; i<nitems; i++) { + deserialize_untagged_u8(&u8, sb); + pmt_u8vector_set(vec, i, u8); + } + return vec; + } + case(UVI_S8): + { + pmt_t vec = pmt_make_s8vector(nitems, 0); + for(uint32_t i=0; i<nitems; i++) { + deserialize_untagged_u8(&u8, sb); + pmt_s8vector_set(vec, i, u8); + } + return vec; + } + case(UVI_U16): + { + pmt_t vec = pmt_make_u16vector(nitems, 0); + for(uint32_t i=0; i<nitems; i++) { + deserialize_untagged_u16(&u16, sb); + pmt_u16vector_set(vec, i, u16); + } + return vec; + } + case(UVI_S16): + { + pmt_t vec = pmt_make_s16vector(nitems, 0); + for(uint32_t i=0; i<nitems; i++) { + deserialize_untagged_u16(&u16, sb); + pmt_s16vector_set(vec, i, u16); + } + return vec; + } + case(UVI_U32): + { + pmt_t vec = pmt_make_u32vector(nitems, 0); + for(uint32_t i=0; i<nitems; i++) { + deserialize_untagged_u32(&u32, sb); + pmt_u32vector_set(vec, i, u32); + } + return vec; + } + case(UVI_S32): + { + pmt_t vec = pmt_make_s32vector(nitems, 0); + for(uint32_t i=0; i<nitems; i++) { + deserialize_untagged_u32(&u32, sb); + pmt_s32vector_set(vec, i, u32); + } + return vec; + } + case(UVI_U64): + { + pmt_t vec = pmt_make_u64vector(nitems, 0); + for(uint32_t i=0; i<nitems; i++) { + deserialize_untagged_u64(&u64, sb); + pmt_u64vector_set(vec, i, u64); + } + return vec; + } + case(UVI_S64): + { + pmt_t vec = pmt_make_s64vector(nitems, 0); + for(uint32_t i=0; i<nitems; i++) { + deserialize_untagged_u64(&u64, sb); + pmt_s64vector_set(vec, i, u64); + } + return vec; + } + case(UVI_F32): + { + pmt_t vec = pmt_make_f32vector(nitems, 0); + for(uint32_t i=0; i<nitems; i++) { + deserialize_untagged_f64(&f64, sb); + pmt_f32vector_set(vec, i, static_cast<float>(f64)); + } + return vec; + } + case(UVI_F64): + { + pmt_t vec = pmt_make_f64vector(nitems, 0); + for(uint32_t i=0; i<nitems; i++) { + deserialize_untagged_f64(&f64, sb); + pmt_f64vector_set(vec, i, f64); + } + return vec; + } + case(UVI_C32): + { + pmt_t vec = pmt_make_c32vector(nitems, 0); + for(uint32_t i=0; i<nitems; i++) { + std::complex<float> c; + deserialize_untagged_f64(&f64, sb); + c.real(static_cast<float>(f64)); + deserialize_untagged_f64(&f64, sb); + c.imag(static_cast<float>(f64)); + pmt_c32vector_set(vec, i, c); + } + return vec; + } + + case(UVI_C64): + { + pmt_t vec = pmt_make_c64vector(nitems, 0); + for(uint32_t i=0; i<nitems; i++) { + std::complex<double> c; + deserialize_untagged_f64(&f64, sb); + c.real(f64); + deserialize_untagged_f64(&f64, sb); + c.imag(f64); + pmt_c64vector_set(vec, i, c); + } + return vec; + } + + default: + throw pmt_exception("pmt_deserialize: malformed input stream, tag value = ", + pmt_from_long(tag)); + } + } + + case PST_DICT: case PST_COMMENT: throw pmt_notimplemented("pmt_deserialize: tag value = ", pmt_from_long(tag)); diff --git a/gruel/src/lib/pmt/unv_template.cc.t b/gruel/src/lib/pmt/unv_template.cc.t index 566168c3d6..f74397421e 100644 --- a/gruel/src/lib/pmt/unv_template.cc.t +++ b/gruel/src/lib/pmt/unv_template.cc.t @@ -87,6 +87,13 @@ pmt_init_@TAG@vector(size_t k, const @TYPE@ *data) return pmt_t(new pmt_@TAG@vector(k, data)); } +pmt_t +pmt_init_@TAG@vector(size_t k, const std::vector< @TYPE@ > &data) +{ + + return pmt_t(new pmt_@TAG@vector(k, &data[0])); +} + @TYPE@ pmt_@TAG@vector_ref(pmt_t vector, size_t k) { diff --git a/gruel/src/python/qa_pmt.py b/gruel/src/python/qa_pmt.py index 1ef4fed15b..59a5725fc4 100755 --- a/gruel/src/python/qa_pmt.py +++ b/gruel/src/python/qa_pmt.py @@ -21,22 +21,82 @@ # import unittest -import pmt_swig +import pmt_swig as pmt class test_gruel_pmt(unittest.TestCase): def test01 (self): - a = pmt_swig.pmt_intern("a") - b = pmt_swig.pmt_from_double(123765) - d1 = pmt_swig.pmt_make_dict() - d2 = pmt_swig.pmt_dict_add(d1, a, b) - pmt_swig.pmt_print(d2) + a = pmt.pmt_intern("a") + b = pmt.pmt_from_double(123765) + d1 = pmt.pmt_make_dict() + d2 = pmt.pmt_dict_add(d1, a, b) + pmt.pmt_print(d2) def test02 (self): const = 123765 - x_pmt = pmt_swig.pmt_from_double(const) - x_int = pmt_swig.pmt_to_double(x_pmt) + x_pmt = pmt.pmt_from_double(const) + x_int = pmt.pmt_to_double(x_pmt) self.assertEqual(x_int, const) + def test03(self): + v = pmt.pmt_init_f32vector(3, [11, -22, 33]) + s = pmt.pmt_serialize_str(v) + d = pmt.pmt_deserialize_str(s) + self.assertTrue(pmt.pmt_equal(v, d)) + + def test04(self): + v = pmt.pmt_init_f64vector(3, [11, -22, 33]) + s = pmt.pmt_serialize_str(v) + d = pmt.pmt_deserialize_str(s) + self.assertTrue(pmt.pmt_equal(v, d)) + + def test05(self): + v = pmt.pmt_init_u8vector(3, [11, 22, 33]) + s = pmt.pmt_serialize_str(v) + d = pmt.pmt_deserialize_str(s) + self.assertTrue(pmt.pmt_equal(v, d)) + + def test06(self): + v = pmt.pmt_init_s8vector(3, [11, -22, 33]) + s = pmt.pmt_serialize_str(v) + d = pmt.pmt_deserialize_str(s) + self.assertTrue(pmt.pmt_equal(v, d)) + + def test07(self): + v = pmt.pmt_init_u16vector(3, [11, 22, 33]) + s = pmt.pmt_serialize_str(v) + d = pmt.pmt_deserialize_str(s) + self.assertTrue(pmt.pmt_equal(v, d)) + + def test08(self): + v = pmt.pmt_init_s16vector(3, [11, -22, 33]) + s = pmt.pmt_serialize_str(v) + d = pmt.pmt_deserialize_str(s) + self.assertTrue(pmt.pmt_equal(v, d)) + + def test09(self): + v = pmt.pmt_init_u32vector(3, [11, 22, 33]) + s = pmt.pmt_serialize_str(v) + d = pmt.pmt_deserialize_str(s) + self.assertTrue(pmt.pmt_equal(v, d)) + + def test10(self): + v = pmt.pmt_init_s32vector(3, [11, -22, 33]) + s = pmt.pmt_serialize_str(v) + d = pmt.pmt_deserialize_str(s) + self.assertTrue(pmt.pmt_equal(v, d)) + + def test11(self): + v = pmt.pmt_init_c32vector(3, [11 + -101j, -22 + 202j, 33 + -303j]) + s = pmt.pmt_serialize_str(v) + d = pmt.pmt_deserialize_str(s) + self.assertTrue(pmt.pmt_equal(v, d)) + + def test12(self): + v = pmt.pmt_init_c64vector(3, [11 + -101j, -22 + 202j, 33 + -303j]) + s = pmt.pmt_serialize_str(v) + d = pmt.pmt_deserialize_str(s) + self.assertTrue(pmt.pmt_equal(v, d)) + if __name__ == '__main__': unittest.main() diff --git a/gruel/src/swig/pmt_swig.i b/gruel/src/swig/pmt_swig.i index b1628c9983..b658571671 100644 --- a/gruel/src/swig/pmt_swig.i +++ b/gruel/src/swig/pmt_swig.i @@ -38,6 +38,27 @@ //load generated python docstrings %include "pmt_swig_doc.i" +%include <std_complex.i> +%include <std_vector.i> +%include <stl.i> + +namespace std { + %template() vector<unsigned char>; + %template() vector<uint8_t>; + %template() vector<char>; + %template() vector<int8_t>; + %template() vector<short>; + %template() vector<uint16_t>; + %template() vector<int16_t>; + %template() vector<int>; + %template() vector<int32_t>; + %template() vector<uint32_t>; + %template() vector<float>; + %template() vector<double>; + %template() vector< std::complex<float> >; + %template() vector< std::complex<double> >; +}; + //////////////////////////////////////////////////////////////////////// // Language independent exception handler //////////////////////////////////////////////////////////////////////// @@ -380,16 +401,17 @@ pmt_t pmt_make_f64vector(size_t k, double fill); pmt_t pmt_make_c32vector(size_t k, std::complex<float> fill); pmt_t pmt_make_c64vector(size_t k, std::complex<double> fill); -pmt_t pmt_init_u8vector(size_t k, const uint8_t *data); -pmt_t pmt_init_s8vector(size_t k, const int8_t *data); -pmt_t pmt_init_u16vector(size_t k, const uint16_t *data); -pmt_t pmt_init_s16vector(size_t k, const int16_t *data); -pmt_t pmt_init_u32vector(size_t k, const uint32_t *data); -pmt_t pmt_init_s32vector(size_t k, const int32_t *data); -pmt_t pmt_init_f32vector(size_t k, const float *data); -pmt_t pmt_init_f64vector(size_t k, const double *data); -pmt_t pmt_init_c32vector(size_t k, const std::complex<float> *data); -pmt_t pmt_init_c64vector(size_t k, const std::complex<double> *data); + +pmt_t pmt_init_u8vector(size_t k, const std::vector<uint8_t> &data); +pmt_t pmt_init_s8vector(size_t k, const std::vector<int8_t> &data); +pmt_t pmt_init_u16vector(size_t k, const std::vector<uint16_t> &data); +pmt_t pmt_init_s16vector(size_t k, const std::vector<int16_t> &data); +pmt_t pmt_init_u32vector(size_t k, const std::vector<uint32_t> &data); +pmt_t pmt_init_s32vector(size_t k, const std::vector<int32_t> &data); +pmt_t pmt_init_f32vector(size_t k, const std::vector<float> &data); +pmt_t pmt_init_f64vector(size_t k, const std::vector<double> &data); +pmt_t pmt_init_c32vector(size_t k, const std::vector<std::complex<float> > &data); +pmt_t pmt_init_c64vector(size_t k, const std::vector<std::complex<double> > &data); uint8_t pmt_u8vector_ref(pmt_t v, size_t k); int8_t pmt_s8vector_ref(pmt_t v, size_t k); |