USING TMS320C80 SDB IN MULTIMEDIA IMAGING APPLICATIONS
PFLIEGEL Péter, SZANCSIK Endre, CSÁNYI Sándor, MOHOS Tibor
Department of Telecommunications, Technical University of Budapest
Abstract: The paper describes SW environment, the way of its use on the 'C80 Software Development Board and the programs developed with the help of these tools. One of the programs serves capture and processing of captured frames while the other implements a video recorder allowing to record very slow motions or -to certain extent- act as a stroboscope. In the paper, the authors focus on the SW development procedure and give some examples of application, as well.
1. Introduction
With the growing bitrate on the Internet and enhanced compression techniques used in multimedia applications, users' demand to meet more and more image and video information at sites of their interest is quite obvious. At the beginning, still and animated icons and different figures made by commercially available drawing software were used to satisfy this demand. Nowadays, due to the mass spread of scanners and digital cameras, 'real things' can also be captured easily and displayed on the Web as still two-dimensional images. The 'lifecams' offer seemingly something different, however, (because of the expected low throughput of the transmission), in fact they strongly 'undersample' the moving reality just updating time-to-time an image showing some interesting place of the world.
Recently a series of low-cost PCTV boards have appeared on the market enabling even the non-professional Web page designers to enrich their sites with images or short video clips taken from TV programs or from their own camera records. Unfortunately such a board comes only with limited SW package enabling not more that was mentioned above. On the top of that, the lack of SW development tools and strongly application-oriented board design makes writing any further application impossible.
At the beginning of the current project, our conditions were quite opposite: we had a powerful, flexible hardware, full SW development environment and every user's guide necessary to know how to use them, on the other hand, it was just a diagnostic program showing the board's functionality without any possibility to even save the image displayed on the screen.
In the following, we briefly describe the HW environment of the project, then we focus on the way we used the given SW tools and pre-written library files, finally we describe the programs that have been developed and suggest some examples of application.
2. HW Environment
The SDB is plugged into a standard 133 MHz Pentium PC equipped with 48 MB RAM and 1.6 GB HD. The SDB video input is fed by PAL signal of a SONY Hi-8 camcorder. An additional display is connected to the SDB's VGA output so that the program debugging can happen independently of the result which can be observed simultaneously on the host PC display.
The analog video signal incoming into the SDB is first digitized into an 8-bit CBS format (fs=29.5 MHz) and then decoded and scaled to a 24-bit 8:8:8 RGB format. The scaled data are written into a 512x64 bit FIFO from where they are loaded line by line to the SDB data bus and written under the control of a memory controller either to the 2 MB VRAM which drives the display block or to the 8 MB DRAM. The SDB memory contents can be accessed by the host via a 64x32 bit FIFO of the PCI controller block.
3. The software approach
The SDB software package and the 'C80 code generation tools contain the necessary tools to develop SDB-based applications running on the host and on the 'C80 master processor (MP). The two area of code development are theoretically independent, e.g. there are a host applications based only on the SDB API and don't require the direct programming of the 'C80 and also a wide range of 'C80 applications use only the SDB peripherals and don't interact with the host. Practically, however, to meet the requirements of a host application, a specific 'C80 program is needed in most cases. In this paper it will be explained, how the applications discussed in the next two sections were developed as a host and a 'C80 program working together.
The SDB and 'C80 software package consists of the SDB Windows NT device driver, 'C80 code generation tools, the SDB API and several 'C80 libraries, SDB utilities (debuggers, SDBShell, SDBDiags), and some example applications. The 'C80 code generation tools are the optimizing MP/PP C compilers, MP/PP assembler and the 'C80 linker. The SDB API (sdbapi.h, .lib, .dll) is essential for developing a host application, because it allows accessing the device driver, and so the communication with the 'C80 command server or the programming certain SDB peripheral registers. Beside the standard C libraries several other function libraries help the 'C80 code development, e.g. a library that creates a multitasking environment (multitasking executive) on the MP, or another 'C80 specific library, called the MVP run-time support library. The SDB embedded library (sdbembed.h, .lib) is a collection of routines that handle the SDB peripherals. The 'C80 debuggers allow MP or PP code debugging on C or assembly level. The SDBShell provides a command line interface to access the SDB resources, while with SDBDiags the main functions of the SDB can be tested. Some example applications played bigger role in the project: one that demonstrates video capture and display (viddemo), and a pair of programs (hostdemo-server) that are dealing with host-SDB communication.
To develop a host application, Microsoft Visual C++ builder is also needed. This generates the application executable from the C/C++ source modules, GUI resources (dialogs, icons, etc.), and many libraries, including the SDB API library. Executable obtained from the above building process can be used with the sdbapi.dll. The API allows the application to interface the device driver, which is the primary way to access the SDB (see Figure 1-a). According to Figure 1-a, host relies only on the basic server, which is loaded by the device driver when it is started. As mentioned above, in most cases an application dependent 'C80 server is needed, which can be developed like any other 'C80 application using the 'C80 libraries and the 'C80 code generation tools. The operation of such a 'C80 program is depicted in Figure 1-b. Two of the 'C80 libraries have been most important in the current project: the multitasking executive library which allows the programming of a server task that meets the needs of the host application, and the SDB embedded library.
Figure 1. SDB Software Development Process
The applications developed by our project also load and start their own 'C80 programs, built in the way described above. One reason for this is that any transfer through the SDB FIFO while video capture packet transfers are running can stop the 'C80 server (the effect is described in the SDB errata documents). Since the continuous video capture is important, we solved this problem with tasks which are less dependent on the host. Another, even more important reason is that creating a video process task (in a similar way like in the viddemo) provides an efficient way of programming the video capture and display processes. Moreover, a server task that cooperates with the video process task allows the transfer of image data to the host. Some simple image processing was also added to the server.
On the top of Figure 1-c, the operation of the host application is shown as object classes and their relations. For reasonable partitioning of class responsibilities, the SDB is accessed from a single class. The operation of that class corresponds exactly to the 'C80 program, described in the previous paragraph. It is also visible on the figure, that not only the server is accessed from this class, but also the SDB peripherals. This is because during any transfer through the SDB FIFO, like in most of the cases when communicating with the server, the video capture packet transfers should be disabled. This is one of bottlenecks in SDB-to-host video throughput (see also Table 1.).
4. The AVI Saver
Starting the AVI Saver, the moving image taken from the camera can be seen on the host PC display and the user can make the necessary camera adjustments and set the image size and framerate before a record is made. Low framerates are for convenience interpreted as 'Time between frames'. This value can be set very precisely (with the resolution of 1 s) so that the timing of slow-process records is quite accurate even up to several hours.
Because of bottlenecks in SDB-to-host data transfer and limited speed in HD access, frames having greater size can be recorded only with lower framerates. The reasons and the actual delays causing bottlenecks are given in Table 1.
Table 1. Maximum framerate as the function of the image size
Image size PCI transfer Format HD oper. Others Total Max. framerate [pixels] [ms] conv. [ms] [ms] [ms] ~[ms] [1/s] 640*480 170 110 400 5 700 1.39 320*240 35 24 90 5 150 6.25 160*120 9 6 21 5 42 12.5 80*60 3 1.5 5 5 15 25
5. The Picture Saver
The program serves for capture and processing still images taken from the camcorder. It can be used as a digital camera with the advantage of keeping up to 35 pictures simultaneously on the clipboard, hence the user can easily compare the shots and save the best one. Besides conventional image adjustments (size, lightness, saturation contrast, flip, mirror, etc.) and some more sophisticated procedures (3D rotation, LP-filtering, edge enhancement) the real strength of the program lies in easy creation of animated GIF images which -compared to uncompressed AVI clips- consume much less space. Making snapshots of different phases of a motion, the pictures can be concatenated in animated GIF immediately within the program.
6. Examples of application
The application possibilities are practically unlimited and depend primarily on the fantasy of the user. We think that the applications in educational software are of prime importance. For instance, we used the AVI Saver as stroboscope to show the vibrations of a loudspeaker diaphragm. Fixing the camcorder on a turntable, it is quite easy to make look-around animated GIF or panoramic AVI clips or use the record for today's popular 'surround' view. The low-framerate record option can be used for animation of low-speed chemical or biological processes, such as observing the change of condition (reaching the freezing/melting or condensing/boiling point), dissolution of a solid material in a liquid, growth of a plant, birth of a living creature, behavior of a person during the sleep, weather changes such formation of a storm, sunrise, sunset, etc.
At the time the paper was written we hoped to record the total solar eclipse and squeeze the almost 3-hour long process into some seconds. By the time we present this paper on the workshop, we will see whether we were successful or not.
References:
[1] Szancsik Endre, Csányi Sándor, Mohos Tibor: Digitális jelfeldolgozó kártya alkalmazása képi multimédia anyagok elõállításában. Three Diploma works, in Hungarian, kons. Peter Pfliegel, Dept. of Telecomm., TUB. Budapest, May 1999.
[2,3] Texas Instruments: TMS320C80 SDB Technical Reference and Programmer's Guide
[4,5,6,7] Texas Instruments: TMS320C80 CG, ME, TC, MP User's Guides.
