Conference system having an audio manager using local and remote audio stream state machines for providing audio control functions during a conference session

Abstract

The audio manager provides an interface between one or more upper-level conferencing drivers of the conferencing system and one or more lower-level audio drivers of the conferencing system to isolate the conferencing drivers from the audio drivers. The audio manager is adapted to perform a plurality of functions called by the conferencing drivers. The audio manager comprises a local audio stream state machine and a remote audio stream state machine.

Claims

What is claimed is:

1. An audio manager for a computer-based conferencing system, wherein:

the audio manager provides an interface between one or more upper-level conferencing drivers of the conferencing system and one or more lower-level audio drivers of the conferencing system to isolate the conferencing drivers from the audio drivers;

the audio manager comprises a local audio stream state machine for controlling the audio drivers for capturing a local audio stream and for transmitting the local audio stream to the conference drivers, wherein the local audio stream state machine comprises

(a) a local audio stream error state;

(b) a local audio stream initialization state;

(c) a local audio stream open state;

(d) a local audio stream capture state; and

(e) a local audio stream linkout state;

the audio manager comprises a remote audio stream state machine for controlling the audio drivers for receiving a remote audio stream from the conference drivers and for playing the remote audio stream locally, wherein the remote audio stream state machine comprises:

(a) a remote audio stream error state;

(b) a remote audio stream initialization state;

(c) a remote audio stream open state;

(d) a remote audio stream linkin state; and

(e) a remote audio stream play state; and

the audio manager is adapted to perform a plurality of functions called by the conferencing drivers, wherein the plurality of functions comprises:

a first function for retrieving the number of different audio managers installed in the conferencing system;

a second function for opening a local or remote audio stream with the audio drivers in accordance with the local or remote audio stream state machine, respectively;

a third function for controlling capturing of the local audio stream with the audio drivers in accordance with the local audio stream state machine;

a fourth function for controlling monitoring of the captured local audio stream with the audio drivers in accordance with the local audio stream state machine;

a fifth function for controlling linking between a network and the captured local audio stream with the audio drivers in accordance with the local audio stream state machine;

a sixth function for controlling linking between the network and the remote audio stream with the audio drivers in accordance with the remote audio stream state machine;

a seventh function for controlling playing of the remote audio stream with the audio drivers in accordance with the remote audio stream state machine;

an eighth function for returning a packet number for a current audio packet of the local or remote audio stream with the audio drivers in accordance with the local or remote audio stream state machine respectively, respectively;

a ninth function for controlling muting of the local or remote audio stream with the audio drivers in accordance with the local or remote audio stream state machine respectively;

a tenth function for adjusting characteristics of the local or remote audio stream with the audio drivers in accordance with the local or remote audio stream state machine, respectively;

an eleventh function for returning status of the local or remote audio stream with the audio drivers in accordance with the local or remote audio stream state machine, respectively;

a twelfth function for returning capabilities of the audio manager;

a thirteenth function for registering an audio stream monitor;

a fourteenth function for closing the local or remote audio stream with the audio drivers in accordance with the local or remote audio stream state machine, respectively; and

a fifteenth function for shutting down the audio manager and the audio drivers.

2. The audio manager of claim 1, wherein:

the first function is an AGetNumDevs function;

the second function is an AOpen function;

the third function is an ACapture function;

the fourth function is an AMonitor function;

the fifth function is an ALinkOut function;

the sixth function is an ALinkIn function;

seventh function is an APlay function;

the eighth function is an APacketNumber function;

the ninth function is an AMute function;

the tenth function is an ACntl function;

the eleventh function is an AGetInfo function;

the twelfth function is an AGetDevCaps function;

the thirteenth function is an ARegisterMonitor function;

the fourteen function is an AClose function; and

the fifteenth function is an AShutdown function.

3. The audio manager of claim 1, wherein:

calling the second function for the local audio stream changes the local audio stream initialization state to the local audio stream open state;

calling the third function for the local audio stream changes between the local audio stream open state and the local audio stream capture state;

calling the fifth function for the local audio stream changes between the local audio stream capture state and the local audio stream linkout state;

calling the fourteenth function for the local audio stream changes any other local audio stream state to the local audio stream initialization state;

calling the second function for the remote audio stream changes the remote audio stream initialization state to the remote audio stream open state;

calling the sixth function for the remote audio stream changes between the remote audio stream open state and the remote audio stream linkin state;

calling the seventh function for the remote audio stream changes between the remote audio stream linkin state and the remote audio stream play state; and

calling the fourteenth function for the remote audio stream changes any other remote audio stream state to the remote audio stream initialization state.

4. The audio manager of claim 3, wherein:

calling the fourth, eighth, ninth, tenth, eleventh, twelfth, thirteenth, or fifteenth function for the local audio stream leaves the local audio stream state unchanged; and

calling the eighth, ninth, tenth, eleventh, twelfth, thirteenth, or fifteenth function for the remote audio stream leaves the remote audio stream state unchanged.

Description

MICROFICHE APPENDIX

Appendices A through E are included in a microfiche appendix submitted as a part of this application. The microfiche appendix has one (1) total microfiche sheets and fifty-two (52) total frames.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to audio/video conferencing and, in particular, to systems for real-time audio, video, and data conferencing in windowed environments on personal computer systems.

2. Description of the Related Art

It is desirable to provide real-time audio, video, and data conferencing between personal computer (PC) systems operating in windowed environments such as those provided by versions of Microsoft.RTM. Windows.TM. operating system. There are difficulties, however, with providing real-time conferencing in non-real-time windowed environments.

It is accordingly an object of this invention to overcome the disadvantages and drawbacks of the known art and to provide real-time audio, video, and data conferencing between PC systems operating in non-real-time windowed environments.

It is a particular object of the present invention to provide real-time audio, video, and data conferencing between PC systems operating under a Microsoft.RTM. Windows.TM. operating system.

Further objects and advantages of this invention will become apparent from the detailed description of a preferred embodiment which follows.

SUMMARY OF THE INVENTION

The present invention comprises an audio manager for a computer-based conferencing system. According to a preferred embodiment, the audio manager provides an interface between one or more upper-level conferencing drivers of the conferencing system and one or more lower-level audio drivers of the conferencing system to isolate the conferencing drivers from the audio drivers. The audio manager is adapted to perform a plurality of functions called by the conferencing drivers. The plurality of functions comprises a first function for retrieving the number of different audio managers installed in the conferencing system; a second function for opening a local or remote audio stream; a third function for controlling capturing of the local audio stream; a fourth function for controlling monitoring of the captured local audio stream; a fifth function for controlling linking between a network and the captured local audio stream; a sixth function for controlling linking between the network and the remote audio stream; a seventh function for controlling playing of the remote audio stream; an eighth function for returning a packet number for a current audio packet of the local or remote audio stream; a ninth function for controlling muting of the local or remote audio stream; a tenth function for adjusting characteristics of the local or remote audio stream; an eleventh function for returning status of the local or remote audio stream; a twelfth function for returning capabilities of the audio manager; a thirteenth function for registering an audio stream monitor; a fourteenth function for closing the local or remote audio stream; and a fifteenth function for shutting down the audio manager and the audio drivers.

The present invention also comprises an audio manager for a computer-based conferencing system. According to a preferred embodiment, the audio manager provides an interface between one or more upper-level conferencing drivers of the conferencing system and one or more lower-level audio drivers of the conferencing system to isolate the conferencing drivers from the audio drivers. The audio manager comprises a local audio stream state machine and a remote audio stream state machine.

BRIEF DESCRIPTION OF THE DRAWINGS

Other objects, features, and advantages of the present invention will become more fully apparent from the following detailed description of the preferred embodiment, the appended claims, and the accompanying drawings in which:

FIG. 1 is a block diagram representing real-time point-to-point audio, video, and data conferencing between two PC systems, according to a preferred embodiment of the present invention;

FIG. 2 is a block diagram of the hardware configuration of the conferencing system of each PC system of FIG. 1;

FIG. 3 is a block diagram of the hardware configuration of the video board of the conferencing system of FIG. 2;

FIG. 4 is a block diagram of the hardware configuration of the audio/comm board of the conferencing system of FIG. 2;

FIG. 5 is a block diagram of the software configuration of the conferencing system of each PC system of FIG. 1;

FIG. 6 is a block diagram of a preferred embodiment of the hardware configuration of the audio/comm board of FIG. 4;

FIG. 7 is a block diagram of the conferencing interface layer between the conferencing applications of FIG. 5, on one side, and the comm, video, and audio managers of FIG. 5, on the other side;

FIG. 8 is a representation of the conferencing call finite state machine (FSM) for a conferencing session between a local conferencing system (i.e., caller) and a remote conferencing system (i.e., callee);

FIG. 9 is a representation of the conferencing stream FSM for each conferencing system participating in a conferencing session;

FIG. 10 is a representation of the video FSM for the local video stream and the remote video stream of a conferencing system during a conferencing session;

FIG. 11 is a block diagram of the software components of the video manager of the conferencing system of FIG. 5;

FIG. 12 is a representation of a sequence of N walking key frames;

FIG. 13 is a representation of the audio FSM for the local audio stream and the remote audio stream of a conferencing system during a conferencing session;

FIG. 14 is a block diagram of the architecture of the audio subsystem of the conferencing system of FIG. 5;

FIG. 15 is a block diagram of the interface between the audio task of FIG. 5 and the audio hardware of audio/comm board of FIG. 2;

FIG. 16 is a block diagram of the interface between the audio task and the comm task of FIG. 5.

FIG. 17 is a block diagram of the comm subsystem of the conferencing system of FIG. 5,

FIG. 18 is a block diagram of the comm subsystem architecture for two conferencing systems of FIG. 5 participating in a conferencing session;

FIG. 19 is a representation of the comm subsystem application FSM for a conferencing session between a local site and a remote site;

FIG. 20 is a representation of the comm subsystem connection FSM for a conferencing session between a local site and a remote site;

FIG. 21 is a representation of the comm subsystem control channel handshake FSM for a conferencing session between a local site and a remote site;

FIG. 22 is a representation of the comm subsystem channel establishment FSM for a conferencing session between a local site and a remote site;

FIG. 23 is a representation of the comm subsystem processing for a typical conferencing session between a caller and a callee;

FIG. 24 is a representation of the structure of a video packet as sent to or received from the comm subsystem of the conferencing system of FIG. 5;

FIG. 25 is a representation of the compressed video bitstream for the conferencing system of FIG. 5;

FIG. 26 is a representation of a compressed audio packet for the conferencing system of FIG. 5;

FIG. 27 is a representation of the reliable transport comm packet structure;

FIG. 28 is a representation of the unreliable transport comm packet structure;

FIG. 29 are diagrams indicating typical connection setup and teardown sequences;

FIGS. 30 and 31 are diagrams of the architecture of the audio/comm board;

FIG. 32 is a diagram of the audio/comm board environment;

FIG. 33 is a flow diagram of the on-demand application invocation processing of the conferencing system of FIG. 5;

FIG. 34 is a flow diagram of an example of the processing implemented within the conferencing system of FIG. 5 to manage two conferencing applications in a single conferencing session with a remote conferencing system;

FIG. 35 represents the flow of bits between two remote high-resolution counters used to maintain clock values over a conferencing network;

FIG. 36 is a flow diagram of the processing of the conferencing system of FIG. 1 to control the flow of signals over reliable channels;

FIG. 37 is a flow diagram of the preemptive priority-based transmission processing implemented by the communications subsystem of the conferencing system of FIG. 1;

FIG. 38 is a state diagram for the complete rate negotiation processing, according to a preferred embodiment of the present invention;

FIG. 39 is a stat e diagram for the rate negotiation processing for a called node during a 64 KBPS upgrade;

FIG. 40 is a state diagram for the rate negotiation processing for a calling node during a 64 KBPS upgrade; and

FIG. 41 is a state diagram for the rate negotiation processing in loopback mode during a 64 KBPS upgrade.

DESCRIPTION OF THE PREFERRED EMBODIMENT(S)

Point-to-Point Conferencing Network

Referring now to FIG. 1, there is shown a block diagram representing real-time point-to-point audio, video, and data conferencing between two PC systems, according to a preferred embodiment of the present invention. Each PC system has a conferencing system 100, a camera 102, a microphone 104, a monitor 106, and a speaker 108. The conferencing systems communicate via an integrated services digital network (ISDN) 110. Each conferencing system 100 receives, digitizes, and compresses the analog video signals generated by camera 102 and the analog audio signals generated by microphone 104. The compressed digital video and audio signals are transmitted to the other conferencing system via ISDN 110, where they are decompressed and converted for play on monitor 106 and speaker 108, respectively. In addition, each conferencing system 100 may generate and transmit data signals to the other conferencing system 100 for play on monitor 106. In a preferred embodiment, the video and data signals are displayed in different windows on monitor 106. Each conferencing system 100 may also display the locally generated video signals in a separate window.

Camera 102 may be any suitable camera for generating NSTC or PAL analog video signals. Microphone 104 may be any suitable microphone for generating analog audio signals. Monitor 106 may be any suitable monitor for displaying video and graphics images and is preferably a VGA monitor. Speaker 108 may be any suitable device for playing analog audio signals and is preferably a head set.

Conferencing System Hardware Configuration

Referring now to FIG. 2, there is shown a block diagram of the hardware configuration of each conferencing system 100 of FIG. 1, according to a preferred embodiment of the present invention. Each conferencing system 100 comprises host processor 202, video board 204, audio/comm board 206, and ISA bus 208.

Referring now to FIG. 3, there is shown a block diagram of the hardware configuration of video board 204 of FIG. 2, according to a preferred embodiment of the present invention. Video board 204 comprises industry standard architecture (ISA) bus interface 310, video bus 312, pixel processor 302, video random access memory (VRAM) device 304, video capture module 306, and video analog-to-digital (A/D) converter 308.

Referring now to FIG. 4, there is shown a block diagram of the hardware configuration of audio/comm board 206 of FIG. 2, according to a preferred embodiment of the present invention. Audio/comm board 206 comprises ISDN interface 402, memory 404, digital signal processor (DSP) 406, and ISA bus interface 408, audio input/output (I/O) hardware 410.

Conferencing System Software Configuration

Referring now to FIG. 5, there is shown a block diagram of the software configuration each conferencing system 100 of FIG. 1, according to a preferred embodiment of the present invention. Video microcode 530 resides and runs on pixel processor 302 of video board 204 of FIG. 3. Comm task 540 and audio task 538 reside and run on DSP 406 of audio/comm board 206 of FIG. 4. All of the other software modules depicted in FIG. 5 reside and run on host processor 202 of FIG. 2.

Video, Audio, and Data Processing

Referring now to FIGS. 3, 4, and 5, audio/video conferencing application 502 running on host processor 202 provides the top-level local control of audio and video conferencing between a local conferencing system (i.e., local site or endpoint) and a remote conferencing system (i.e., remote site or endpoint). Audio/video conferencing application 502 controls local audio and video processing and establishes links with the remote site for transmitting and receiving audio and video over the ISDN. Similarly, data conferencing application 504, also running on host processor 202, provides the top-level local control of data conferencing between the local and remote sites. Conferencing applications 502 and 504 communicate with the audio, video, and comm subsystems using conference manager 544, conferencing application programming interface (API) 506, video API 508, comm API 510, and audio API 512. The functions of conferencing applications 502 and 504 and the APIs they use are described in further detail later in this specification.

During conferencing, audio I/O hardware 410 of audio/comm board 206 digitizes analog audio signals received from microphone 104 and stores the resulting uncompressed digital audio to memory 404 via ISA bus interface 408. Audio task 538, running on DSP 406, controls the compression of the uncompressed audio and stores the resulting compressed audio back to memory 404. Comm task 540, also running on DSP 406, then formats the compressed audio for ISDN transmission and transmits the compressed ISDN-formatted audio to ISDN interface 402 for transmission to the remote site over ISDN 110.

ISDN interface 402 also receives from ISDN 110 compressed ISDN-formatted audio generated by the remote site and stores the compressed ISDN-formatted audio to memory 404. Comm task 540 then reconstructs the compressed audio format and stores the compressed audio back to memory 404. Audio task 538 controls the decompression of the compressed audio and stores the resulting decompressed audio back to memory 404. ISA bus interface then transmits the decompressed audio to audio I/O hardware 410, which digital-to-analog (D/A) converts the decompressed audio and transmits the resulting analog audio signals to speaker 108 for play.

Thus, audio capture/compression and decompression/playback are preferably performed entirely within audio/comm board 206 without going through the host processor. As a result, audio is preferably continuously played during a conferencing session regardless of what other applications are running on host processor 202.

Concurrent with the audio processing, video A/D converter 308 of video board 204 digitizes analog video signals received from camera 102 and transmits the resulting digitized video to video capture module 306. Video capture module 306 decodes the digitized video into YUV color components and delivers uncompressed digital video bitmaps to VRAM 304 via video bus 312. Video microcode 530, running on pixel processor 302, compresses the uncompressed video bitmaps and stores the resulting compressed video back to VRAM 304. ISA bus interface 310 then transmits via ISA bus 208 the compressed video to video/host interface 526 running on host processor 202.

Video/host interface 526 passes the compressed video to video manager 516 via video capture driver 522. Video manager 516 calls audio manager 520 using audio API 512 for synchronization information. Video manager 516 then time-stamps the video for synchronization with the audio. Video manager 516 passes the time-stamped compressed video to communications (comm) manager 518 using comm application programming interface (API) 510. Comm manager 518 passes the compressed video through digital signal processing (DSP) interface 528 to ISA bus interface 408 of audio/comm board 206, which stores the compressed video to memory 404. Comm task 540 then formats the compressed video for ISDN transmission and transmits the ISDN-formatted compressed video to ISDN interface 402 for transmission to the remote site over ISDN 110.

ISDN interface 402 also receives from ISDN 110 ISDN-formatted compressed video generated by the remote site system and stores the ISDN-formatted compressed video to memory 404. Comm task 540 reconstructs the compressed video format and stores the resulting compressed video back to memory 404. ISA bus interface then transmits the compressed video to comm manager 518 via ISA bus 208 and DSP interface 528. Comm manager 518 passes the compressed video to video manager 516 using video API 508. Video manager 516 passes the compressed video to video decode driver 548 for decompression processing. Video decode driver 548 passes the decompressed video to video playback driver 550, which formats the decompressed video for transmission to the graphics device interface (GDI) (not shown) of the Microsoft.RTM. Windows.TM. operating system for eventual display in a video window on monitor 106.

For data conferencing, concurrent with audio and video conferencing, data conferencing application 504 generates and passes data to comm manager 518 using conferencing API 506 and comm API 510. Comm manager 518 passes the data through board DSP interface 532 to ISA bus interface 408, which stores the data to memory 404. Comm task 540 formats the data for ISDN transmission and stores the ISDN-formatted data back to memory 404. ISDN interface 402 then transmits the ISDN-formatted data to the remote site over ISDN 110.

ISDN interface 402 also receives from ISDN 110 ISDN-formatted data generated by the remote site and stores the ISDN-formatted data to memory 404. Comm task 540 reconstructs the data format and stores the resulting data back to memory 404. ISA bus interface 408 then transmits the data to comm manager 518, via ISA bus 208 and DSP interface 528. Comm manager 518 passes the data to data conferencing application 504 using comm API 510 and conferencing API 506. Data conferencing application 504 processes the data and transmits the processed data to Microsoft.RTM. Windows.TM. GDI (not shown) for display in a data window on monitor 106.

Preferred Hardware Configuration for Conferencing System

Referring now to FIG. 6, there is shown a block diagram of a preferred embodiment of the hardware configuration of audio/comm board 206 of FIG. 4. Referring now to FIGS. 30 and 31, there are shown diagrams of the architecture of the audio/comm board. Referring now to FIG. 32, there is shown a diagram of the audio/comm board environment. The description for this section is the same as the description for the section of the same name in U.S. patent application Ser. No. 08/157,694.

Software Architecture for Conferencing System

The software architecture of conferencing system 100 shown in FIGS. 2 and 5 has three layers of abstraction. A computer supported collaboration (CSC) infrastructure layer comprises the hardware (i.e., video board 204 and audio/comm board 206) and host/board driver software (i.e., video/host interface 526 and DSP interface 528) to support video, audio, and comm, as well as the encode method for video (running on video board 204) and encode/decode methods for audio (running on audio/comm board 206). The capabilities of the CSC infrastructure are provided to the upper layer as a device driver interface (DDI).

A CSC system software layer provides services for instantiating and controlling the video and audio streams, synchronizing the two streams, and establishing and gracefully ending a call and associated communication channels. This functionality is provided in an application programming interface (API). This API comprises the extended audio and video interfaces and the communications APIs (i.e., conference manager 544, conferencing API 506, video API 508, video manager 516, video capture driver 522, video decode driver 548, video playback driver 550, comm API 510, comm manager 518. Wave API 514, Wave driver 524, PWave API 552, audio API 512, and audio manager 520).

A CSC applications layer brings CSC to the desktop. The CSC applications may include video annotation to video mail, video answering machine, audio/video/data conferencing (i.e., audio/video conferencing application 502 and data conferencing application 504), and group decision support systems.

Audio/video conferencing application 502 and data conferencing application 504 rely on conference manager 554 and conferencing API 506, which in turn rely upon video API 508, comm API 510, and audio API 512 to interface with video manager 516, comm manager 518, and audio manager 520, respectively. Comm API 510 and comm manager 518 provide a transport-independent interface (TII) that provides communications services to conferencing applications 502 and 504. The communications software of conferencing system 100 may be designed to support different transport mechanisms, such as ISDN, SW56, and LAN (e.g., SPX/IPX, TCP/IP, or NetBIOS). The TII isolates the conferencing applications from the underlying transport layer (i.e., transport-medium-specific DSP interface 528). The TII hides the network/connectivity specific operations. In conferencing system 100, the TII hides the ISDN layer. The DSP interface 528 is hidden in the datalink module (DLM). The TII provides services to the conferencing applications for opening communication channels (within the same session) and dynamically managing the bandwidth. The bandwidth is managed through the transmission priority scheme.

In a preferred embodiment in which conferencing system 100 performs software video decoding, video capture driver 522 is implemented on top of video/host interface 526 (the video driver). In an alternative preferred embodiment in which conferencing system 100 performs hardware video decoding, a video display driver is also implemented on top of video/host interface 526.

The software architecture of conferencing system 100 comprises three major subsystems: video, audio, and communication. The audio and video subsystems are decoupled and treated as "data types" (similar to text or graphics) with conventional operations like open, save, edit, and display. The video and audio services are available to the applications through video-management and audio-management extended interfaces, respectively.

In a preferred embodiment, conferencing system 100 is implemented mostly in the C++ computer language using the Microsoft.RTM. Foundation Classes (MFC) with portions implemented in the C7.0 computer language.

Audio/Video Conferencing Application

Audio/video conferencing application 502 implements the conferencing user interface. Conferencing application 502 is implemented as a Microsoft.RTM. Windows.TM. 3.1 application. One child window will display the local video image and a second child window will display the remote video image. Audio/video conferencing application 502 provides the following services to conferencing system 100:

Manages main message loop.

Performs initialization and registers classes.

Handles menus.

Processes toolbar messages.

Handles preferences.

Handles speed dial setup and selections.

Connects and hang up.

Handles handset window

Handles remote video.

Handles remote video window.

Handles local video.

Handles local video window.

Handles call notification dialog box.

Plays sounds.

Handles address book lookup for caller identification purposes.

Audio/video conferencing application 502 is made up of six main modules:

(1) ABIF.LIB: An address book class wrapper library.

(2) SUBDLG.LIB: A sub-dialog library.

(3) CMDLL.DLL: A conference manager DLL (shown in FIG. 5 as conference manager 544).

(4) CMIF.LIB: A library with high-level interfaces to CMDLL.DLL and other sub-systems.

(5) PSVIDEO.EXE: An audio/video application proper.

(6) PSNOTIFY.EXE: An incoming call notification application (a listener).

CMIF.LIB

The CMIF.LIB CMIF was created to encapsulate several subsystem APIs: conference manager API 544, conferencing API (VCI) 506, audio API 512, and video API 508. It is a set of (mostly) static classes (data and functions). It consists of 3 classes: CCm, CImageSize, and CImageState. CImageSize encapsulates the various sizes that video images can have. CImageState encapsulates the different attributes and drivers that a video image can have. CCm handles everything not covered by CImageSize and CImageState. CCm includes a callback function to handle notification from CMDLL.

The callback function in CCm can be called in the context of another application (namely VTHREAD.EXE of VCI 506). This is the reason that CMIF is a separate library. By making it a separate library the compile flags are set to use smart callbacks in a way that is compatible with MakeProcInstance(). MakeProcInstance() is used on the callback to force the data segment to point appropriately upon entering the callback function.

CCm

The CCm class simplifies calling routines in subsystem APIs. It has the following capabilities: loading and unloading of the subsystem DLLs, registering and unregistering with CMDLL, call support, channel pair support, stream support, CMDLL callback handler, and NO.sub.-- VCI support. CCm assumes that there is one video call. This call can be in various different states: disconnected, connecting, connected, and disconnecting. Tests and assertions are performed throughout the code to verify that operations on a call can be carried out during the current call state.

Loading and Unloading

CCm::Load() loads the various subsystems. It does so by calling LoadLibrary() for each DLL. For each subsystem, after successfully loading the library into memory, CCm::Load() loads a table of function pointers for each subsystem by calling GetProcAddress(). There are macros that allow CCm to call these functions in a manner consistent with implicit linking.

CCm::Unload() unloads the various subsystems. It does so by calling UnloadLibrary() for each DLL.

Registering and Unregistering

CCm::Register() registers CCm::Callback() as a callback function with CMDLL 544. It also registers an application window with CMIF to post messages to. It should be called once upon application startup.

CCm::UnRegister() unregisters with CMDLL. It should be called once upon application shutdown.

CCm::Install() allows an application that is not currently installed in CMDLL's registry to install itself there. This should be called if CCm::Register() fails with a return code of CMRC.sub.-- UNKNOWNAPP. Then, after successfully calling CCm::Install(), call CCm::Register() again.

Call Support

CCm::MakeCall() allows the application to place a video call via CMDLL 544. CCm::MakeCall() takes a ISDN phone number (of the callee) and a name (of the caller). The name is passed to the callee for display in the caption of the remote video window. Upon successful return from CCm::MakeCall(), the call will be in the connecting state.

CCm::HangupCall() hangs the current call up. If there is no call in progress, it returns an error code. Upon successful return from CCm::HangupCall(), the call will be in the disconnecting state.

Channel Pair Support

A channel pair is a bidirectional VCI/TII communication channel. It can be used to send and receive data from a peer application during a video call. Channel pairs are established with a peer application (facilitated through registration) and are referred to by an id (0-31). CMDLL 544 provides this channel pair support to CMIF.

CCm::GetChannelPairInfo() can be called with a channel pair id. It will return information pertaining to the given channel pair or an error code if it failed.

CCm::GetChannelPair() requests a channel pair from CMDLL. The request is completed when CMDLL sends the CMN.sub.-- CHANNELPAIR callback message. This message indicates whether or not the channel was successfully opened. Once a channel pair is opened it needs to have a channel handler registered for it. This is done by calling CCm::RegisterChanHandler() for each channel in the channel pair.

After successfully registering a channel handler for each channel in a channel pair, the channel pair can be used to send and receive data.

CCm::SendData() requests that data be sent on the outbound channel within a channel pair. The request is complete when VCI sends the CHAN.sub.-- DATA.sub.-- SENT callback message to the channel handler.

CCm::ReceiveData() posts a receive buffer to VCI/TII for the inbound channel to receive data into. When data has been successfully received into a buffer for the inbound channel, VCI sends the CHAN.sub.-- RCV.sub.-- COMPLETE callback message to the channel handler. This buffer must not be freed until the receive has completed.

When finished using a channel pair, call CCm::CloseChannelPair() to request that the resources associated with it be freed. The request is complete when CMDLL sends the CMN.sub.-- CLOSECHANNELPAIR callback message. After receiving this message do not attempt to use the channels in that channel pair.

PSVIDEO encapsulates registering a channel handler and sending and receiving data in a class called CChannelPair.

Stream Support

VCI stream groups contain both an audio data stream and a video data stream. They can be either local or remote. Local stream groups are created by calling CCm::CreateLocalStream() and remote stream groups are created by calling CCm::CreateRemoteStream(). Both take as parameters a window in which to play the video, a size to play the video at, a state to play the video in, and flags specifying whether or not to play the video/audio. Local stream groups do not play audio locally, whereas remote stream groups do. There is preferably only local stream group and one remote stream group.

To start sending the local stream group to the peer video conferencing application, call CCm::SendLocalStream(). This should be called only once after the video call has been established and the local stream group has been created.

To start and stop sending during a video call, call CCm::SendLocalStreamMute(). This has the same syntax as the former, but does not stop the audio time stamps from being sent to the peer application. These audio time stamps are preferably continually sent, after they have been started for the life of a video call.

To start and stop playing either the local stream group or the remote stream group, call CCm::PlayLocalStream() or CCm::PlayRemoteStream(), respectively. These also can be used to play a stream at a different size or state than it was previously being played at before (i.e., the local stream group can be played as an IRV video stream or a YUV video stream).

To change the window in which either the local stream group or the remote stream group is being played, call CCm::RedirectLocalVideo() or CCm::RedirectRemoteVideo(), respectively.

To take a snapshot of either stream, call CCm::SnapStream(). It will return a device independent bitmap of the given stream group's current video image. It does not matter what the size or state of the video stream is or whether it is the local or remote stream group.

To control the volume of the remote stream group (remember local streams do not play audio), call CCm::SetWaveVolume().

To switch the remote stream group in and out of open audio (half-duplex speaker phone), call CCm::SetOpenAudio().

To control either stream groups video attributes (color, contrast, brightness, and tint), call CCm::SetColor(), CCm::SetContrast(), CCm::SetBrightness(), and CCm::SetTint(). However, CCm::SetTint() is not preferably supported for the remote stream group.

To adjust the display quality (frequency) of the local stream group, call CCm::SetFRC(). Also, local stream groups can toggle between mirror and camera views and normal and wide angle views.

To toggle these attributes, call CCm::SetMirrorView() and CCm::SetWideAngle(), respectively.

CCm::Redraw() forces the given stream group to redraw itself.

To free the resources allocated for a stream group call CCm::DestroyStream( ).

CMDLL Callback

When registering with CMDLL 544, a callback function is specified. CMIF specifies a procedure instance for CCm::CallBack(). This callback function may be called in the context of another application. This is why MakeProcInstance() is used to generate a procedure instance for the callback. However, smart callbacks (e.g., Microsoft.RTM. C8.0 compiler flag /GA) are not compatible with MakeProcInstance(). Use the old style callbacks (e.g., Microsoft.RTM. C8.0 compiler flag /Gw, /GW or /GA/GEea) instead.

This callback function receives the following types of messages from CMDLL 544: call messages (CMN.sub.-- CALL, CMN.sub.-- HANGUP, CMN.sub.-- REJECTED, CMN.sub.-- PROGRESS), channel pair messages (CMN.sub.-- CHANNELPAIR, CMN.sub.-- CLOSECHANNELPAIR), CMDLL shutdown message (CMN.sub.-- EXIT) and error message (CMN.sub.-- ERROR). For each message sent to this callback, a message is posted to the application window that was given during registration.

When a call message is received, the call state is tested to validate whether the message is valid in the current call state or not. When a message is received that does not correspond to the current call state, a debug message is printed out and in some cases an appropriate action is taken to correct the situation. When a CMN.sub.-- CALL, CMN.sub.-- HANGUP, or CMN.sub.-- REJECTED message is received, the call state is updated to reflect the state diagram shown above.

NO VCI Support

The NO.sub.-- VCI compiler is a user interface (UI) development tool to help UI developers debug and test their code without the subsystems running. Most every function that calls a subsystem has the following code fragment:

    ______________________________________
    // code to handle either case
    #ifindef NO.sub.-- VCI
    // code to handle the case when calling the subsystem
    .
    .
    #else
    // code to handle the case when not calling the subsystem
    .
    .
    .
    #endif
    // code to handle either case
    ______________________________________

    __________________________________________________________________________
    typedef struct tagChanMsgData {
    WORD      wMsg;  // message identifier.
    DWORD     dwMsgParam;
                     // message specific parameter/value.
    DWORD     dwSequence;
                     // sequence number for a series of control packets.
    WORD      wBufferSize;
                     // size of the buffer following the structure.
    } ChanMsgData, FAR* LPCHANMSGDATA;
    __________________________________________________________________________

    __________________________________________________________________________
    cmcall.c
         Contains the icmMakeCall and icmHangupCall procedures that are
         called from
         the CMIF library.
    cmchan.c
         Contains the implementation of channel related procedures specified
         in the
         conference manager API. These are cmGetChannelPair,
         cmCloseChannelPair,
         and cmGetChannelPairInfo.
    cmclntfy.c
         Contains dialog box procedures for the dialogs that are displayed on
         incoming
         calls. There is one dialog for the case when the caller ID matches a
         record in
         the address book, and one dialog for the case when match is not
         found.
    cmdll.c
         Contains the LibMain and WEP procedures. Also contains various
         initialization procedures, including Lib.sub.-- InitializeCf which
         loads VCI.DLL and
         makes the VCI call CF.sub.-- Init to initialize the comm subsystem
         and
         Lib.sub.-- TerminateCf which calls CF.sub.-- UnInit and unloads
         VCI.DLL. This module
         also contains code for registering and unregistering with the
         address book
         services provided by ABDLL.DLL.
    cmmisc.c
         Contains miscellaneous supporting functions used throughout the
         other
         modules in CMDLL.
    cmnotify.c
         Contains callbacks required by VCI. The callback Lib.sub.-- CfCallCal
         lBack
         handles the CFM.sub.-- * messages such as CFM.sub.-- CALL.sub.--
         NTFY and
         CFM.sub.-- CALL.sub.-- HANGUP. The callback Lib.sub.-- CfChanCallBack
          handles VCI
         CFM.sub.-- CHAN.sub.-- * channel messages such as CFM.sub.--
         CHAN.sub.-- ACCEPT.sub.-- NTFY and
         CFM.sub.-- CHAN.sub.-- REJECT.sub.-- NTFY.
    cmreg.c
         Contains the implementation of the conference manager API functions
         cmRegister, cmUnregister, and cmInstall.
    __________________________________________________________________________

    ______________________________________
    typedef struct tagCHANNELPAIR
    HCHAN        hChanIn;  // input (receive) channel
    HCHAN        hChanOut; // output (send) channel
    WORD         wState;   // channel pair state (CPS.sub.-- *)
    CMCHAN.sub.-- INFO
                 cmChanInfo;
                           // channel info struct
    } CHANNELPAIR;
    ______________________________________

    __________________________________________________________________________
    typedef struct tagCMCHAN.sub.-- INFO
    HCHAN     hChanIn;
                    // input (read) channel
    HCHAN     hChanOut;
                    // output (send) channel
    CHAN.sub.-- INFO
              chanInfo;
                    // channel information
    DWORD     dwTransId;
                    // transaction id
    BYTE      byUsageId;
                    // usage id
    BOOL      bOpener;
                    // TRUE if initiator of cmGetChannelPair, else FALSE
    } CMCHAN.sub.-- INFO;
    __________________________________________________________________________

    ______________________________________
    BOOL CallNotifyDlg.sub.-- Do(
    IN HWND hWndOwner,
                   // owner of dialog windows (handset
                   window)
    IN HAB.sub.-- REC hAbRec,
                   // address book record, possibly NULL
    IN LPSTR lpszCallerID
                   // caller ID string from comm subsystem
    ______________________________________

    ______________________________________
    ›Preferences!
    ______________________________________
    AudioPreference=1
                   ; 0 means use default
    WavePath=c:.backslash.myring.wav
                   ; if AudioPreference is 1, use this wave
                   file for incoming calls
    ______________________________________

    __________________________________________________________________________
    CF.sub.-- Init
                Reads in the conferencing configuration parameters from an
                initialization file;
                loads and initializes the software of the comm, video, and
                audio subsystems by
                allocating and building internal data structures; allows the
                application to
                choose between the message and the callback routines to
                return the event
                notifications from the remote site.
    CF.sub.-- MakeCall
                Makes a call to the remote site to establish a connection for
                conferencing. The
                call is performed asynchronously.
    CF.sub.-- AcceptCall
                Accepts call initiated from the remote site based on
                information received in the
                CFM.sub.-- CALL.sub.-- NTFY message as delivered to the
                graphical user interface.
    CF.sub.-- RejectCall
                Rejects incoming call, if appropriate, upon receiving a
                CFM.sub.-- CALL.sub.-- NTFY
                message as delivered to the GUI.
    CF.sub.-- HangupCall
                Hangs up a call that was previously established; releases all
                resources,
                including all types of streams and data structures, allocated
                during the call.
    CF.sub.-- GetCallInfo
                Returns the information about the specified call, including
                its current state.
    CF.sub.-- CapMon
                Starts the capture of analog video signals from the local
                camera and displays
                the video in the local video window which is pre-opened by
                the application.
                This function allows the user to preview his/her appearance
                before sending the
                signals out to the remote site.
    CF.sub.-- PlayRcvd
                Starts the reception and display of remote video signals in
                the remote video
                window, which is pre-opened by the application; starts the
                reception and play
                of remote audio signals through the local speaker.
    CF.sub.-- DestroyStream
                Destroys the specified stream group that was created by
                CF.sub.-- CapMon or
                CF.sub.-- PlayRcvd. As part of the destroy process, all
                operations (e.g.,
                sending/playing) being performed on the stream group will be
                stopped and all
                allocated system resources will be freed.
    CF.sub.-- Mute
                Uses AMute to turn on/off the mute function being performed
                on the audio
                stream of a specified stream group. This function will
                temporarily stop or
                restart the related operations, including playing and
                sending, being performed
                on this stream group. This function may be used to hold
                temporarily one
                audio stream and provide more bandwidth for other streams to
                use.
    CF.sub.-- SnapStream
                Takes a snapshot of the video stream of the specified stream
                group and returns
                a still image (reference) frame to the application buffers
                indicated by the
                hBuffer handle.
    CF.sub.-- ControlStream
                Controls the capture or playback functions of the local or
                remote video and
                audio stream groups.
    CF.sub.-- SendStream
                Uses ALinkOut to pause/unpause audio.
    CF.sub.-- GetStreamInfo
                Returns the current state and the audio video control block
                (AVCB) data
                structure, preallocated by the application, of the specified
                stream groups.
    CF.sub.-- PlayStream
                Stops/starts the playback of the remote audio/video streams
                by calling
                APlay/VPlay.
    CF.sub.-- GetAudVidStream
                Returns the audio and video stream handles for the specified
                stream group.
    CF.sub.-- RegisterChanMgr
                Registers a callback or an application window whose message
                processing
                function will handle notifications generated by network
                channel initialization
                operations. This function is invoked before any CF.sub.--
                OpenChannel calls are
                made.
    CF.sub.-- OpenChannel
                Requests to open a network channel with the peer application.
                The result of
                the action is given to the application by invoking the
                callback routine specified
                by the call to CF.sub.-- RegisterChanMgr. The application
                specifies an ID for this
                transaction. This ID is passed to the callback routine or
                posted in a message.
    CF.sub.-- AcceptChannel
                A peer application can issue CF.sub.-- AcceptChannel in
                response to a
                CFM.sub.-- CHAN.sub.-- OPEN.sub.-- NTFY message that has been
                received. The result of the
                CF.sub.-- AcceptChannel call is a one-way network channel for
                receiving data.
    CF.sub.-- RejectChannel
                This routine rejects a CFM.sub.-- CHAN.sub.-- OPEN.sub.--
                NTFY from the peer.
    CF.sub.-- RegisterChanHandler
                This function registers a callback or an application window
                whose
                message processing function will handle notifications
                generated by TII
                network channel IO activities. The channels that are opened
                will
                receive TII CHAN.sub.-- DATA.sub.-- SENT notifications, and
                the channels that
                are accepted will receive TII CHAN.sub.-- RCV.sub.-- COMPLETE
                notifications.
    CF.sub.-- CloseChannel
                This routine will close a network channel that was opened by
                CF.sub.-- AcceptChannel or CF.sub.-- OpenChannel. The handler
                for this channel is
                automatically de-registered.
    CF.sub.-- SendData
                Send data to peer. If the channel is not reliable and there
                are no receive
                buffers posted on the peer machine, the data will be lost.
    CF.sub.-- RecvData
                Data is received through this mechanism. Normally this call
                is issued in order
                to post receive buffers to the system. When the system has
                received data in
                the given buffers, the Channel Handler will receive the TII
                CHAN.sub.-- RCV.sub.-- COMPLETE notification.
    CF.sub.-- GetChanInfo
                This function will return various statistical information
                about a channel. For
                example: bandwidth information, number of sends/second,
                number of
                receives/second, etc.
    __________________________________________________________________________

    __________________________________________________________________________
    CFM.sub.-- CALL.sub.-- NTFY
                Indicates that a call request initiated from the remote site
                has been
                received.
    CFM.sub.-- PROGRESS.sub.-- NTFY
                Indicates that a call state/progress notification has been
                received from
                the local phone system support.
    CFM.sub.-- ACCEPT.sub.-- NTFY
                Indicates that the remote site has accepted the call request
                issued
                locally. Also sent to the accepting application when
                CF.sub.-- AcceptCall
                completes.
    CFM.sub.-- REJECT.sub.-- NTFY
                Indicates that the remote site has rejected or the local site
                has failed to
                make the call.
    CFM.sub.-- HANGUP.sub.-- NTFY
                Indicates that the local or remote site has hung up the
                call.
    CFM.sub.-- UNINIT.sub.-- NTFY
                Indicates that uninitialization of comm subsystem has
                completed.
    CFM.sub.-- ERROR.sub.-- NTFY
                Indicates that a SESS.sub.-- ERROR was received from comm
                subsystem.
    __________________________________________________________________________

    __________________________________________________________________________
    CCST.sub.-- NULL
               Null State - state of uninitialized caller/callee.
    CCST.sub.-- IDLE
               Idle State - state of caller/callee ready to make/receive
               calls.
    CCST.sub.-- CALLING
               Calling state - state of caller trying to call callee.
    CCST.sub.-- CALLED
               Called state - state of callee being called by caller.
    CCST.sub.-- ACCEPTING
               Accepting state - state of accepting call from caller.
    CCST.sub.-- CONNECTED
               Call state - state of caller and callee during conferencing
               session.
    CCST.sub.-- CLOSING
               A hangup or call cleanup is in progress.
    __________________________________________________________________________

    __________________________________________________________________________
    CSST.sub.-- INIT
            Initialization state - state of local and remote streams after
            CCST.sub.-- CONNECTED state is first reached.
    CSST.sub.-- ACTIVE
            Capture state - state of local stream being captured. Receive
            state - state of
            remote stream being received.
    CSST.sub.-- FAILURE
            Fail state - state of local/remote stream after resource
    __________________________________________________________________________
            failure.