System and method for configuring an instrument to perform measurement functions utilizing conversion of graphical programs into hardware implementations

Abstract

A computer-implemented system and method for generating a hardware implementation of graphical code. The method may operate to configure an instrument to perform measurement functions, wherein the instrument includes a programmable hardware element. The method comprises first creating a graphical program, wherein the graphical program may implement a measurement function. A portion of the graphical program may be converted into a hardware implementation on a programmable hardware element, and a portion may optionally be compiled into machine code for execution by a CPU. The programmable hardware element is thus configured utilizing a hardware description and implements a hardware implementation of at least a portion of the graphical program.

Claims

We claim:

1. A computer-implemented method for configuring an instrument to perform a measurement function, wherein the instrument includes a programmable hardware element, the method comprising:

creating a graphical program, wherein the graphical program implements the measurement function, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;

generating a hardware description based on the graphical program, wherein the hardware description describes a hardware implementation of the graphical program;

configuring the programmable hardware element in the instrument utilizing the hardware description, wherein after said configuring the programmable hardware element implements a hardware implementation of the graphical program;

the instrument acquiring a signal from an external source after said configuring;

the programmable hardware element in the instrument executing to perform the measurement function on the signal.

2. The method of claim 1,

wherein the instrument comprises an image acquisition device coupled to a camera;

wherein the graphical program comprises an image processing function;

wherein the instrument acquiring a signal from an external source comprises the camera acquiring an image of an object and the image acquisition device receiving and storing the image;

wherein the image acquisition device comprises the programmable hardware element;

wherein said executing comprises the programmable hardware element in the image acquisition device executing to perform the image processing function on the image.

3. The method of claim 2,

wherein said executing further comprises:

the programmable hardware element in the image acquisition device determining characteristics of the image after performing the image processing function; and

the programmable hardware element in the image acquisition device performing an operation based on the determined characteristics of the image.

4. The method of claim 1,

wherein the instrument comprises a smart camera;

wherein the instrument acquiring a signal from an external source comprises the smart camera acquiring an image of an object and storing the image; and

wherein the programmable hardware element in the smart camera executes to perform an image processing function on the image.

5. The method of claim 4,

wherein said executing further comprises:

the programmable hardware element in the smart camera determining characteristics of the image after performing the image processing function; and

the programmable hardware element in the smart camera performing an operation based on the determined characteristics of the image.

6. The method of claim 1,

wherein the instrument comprises a smart sensor;

wherein the instrument acquiring a signal from an external source comprises the smart sensor acquiring the signal from the external source; and

wherein the programmable hardware element in the smart sensor executes to perform the measurement function on the signal.

7. The method of claim 1, wherein the smart sensor is one of a temperature sensor, pressure sensor, camera, microphone, flow sensor, position sensor, velocity sensor, acceleration sensor, vibration sensor, electromagnetic sensor, optical sensor, radiation sensor, or environment sensor.

8. The method of claim 1,

wherein the instrument comprises a fieldbus device;

wherein the instrument acquiring a signal from an external source comprises the fieldbus device acquiring the signal from the external source; and

wherein the programmable hardware element in the fieldbus device executes to perform the measurement function on the signal.

9. The method of claim 1,

wherein the instrument comprises a programmable logic controller (PLC);

wherein the instrument acquiring a signal from an external source comprises the PLC acquiring the signal from the external source; and

wherein the programmable hardware element in the PLC executes to perform the measurement function on the signal.

10. The method of claim 1,

wherein the instrument comprises a computer-based instrument, wherein the computer-based instrument is configured as a card inserted into a slot of a computer system;

wherein the instrument acquiring a signal from an external source comprises the computer-based instrument acquiring the signal from the external source; and

wherein the programmable hardware element in the computer-based instrument executes to perform the measurement function on the signal.

11. The method of claim 1,

wherein the instrument comprises a reconfigurable measurement device, wherein the reconfigurable measurement device is configured as a card inserted into a slot of a computer system;

wherein the instrument acquiring a signal from an external source comprises the reconfigurable measurement device acquiring the signal from the external source; and

wherein the programmable hardware element in the reconfigurable measurement device executes to perform the measurement function on the signal.

12. The method of claim 1,

wherein the graphical program includes a user interface portion;

the method further comprising:

presenting the user interface portion on a display during the programmable hardware element in the instrument executing to perform the measurement function on the signal.

13. The method of claim 12,

wherein the user interface portion operates as a front panel for the instrument.

14. The method of claim 12, the method further comprising:

receiving user input to the user interface portion on the display to control the instrument during the programmable hardware element in the instrument executing to perform the measurement function on the signal.

15. The method of claim 12,

wherein the user interface portion includes one or more user interface objects;

wherein said generating includes incorporating a register in the hardware description for at least one of the user interface objects;

wherein the programmable hardware element in the instrument executing to perform the measurement function on the signal includes accessing a register on the programmable hardware element to affect one of said user interface objects.

16. The method of claim 12, further comprising:

compiling the user interface portion into executable code for execution by a processor and storing the executable code in a memory;

the processor executing the executable code from the memory to present the user interface portion on the display.

17. The method of claim 16, wherein the instrument includes the processor and the memory;

wherein the processor in the instrument executes the executable code from the memory to present the user interface portion on the display during the programmable hardware element in the instrument executing to perform the measurement function on the signal.

18. The method of claim 16, wherein the instrument is coupled to a computer system, wherein the computer system includes the processor and the memory;

wherein the computer system executes the executable code from the memory to present the user interface portion on the display during the programmable hardware element in the instrument executing to perform the measurement function on the signal.

19. The method of claim 1, the method further comprising:

performing analog to digital conversion on the signal after said acquiring and prior to said executing.

20. The method of claim 1, wherein the instrument further includes timer/counter logic, the method further comprising:

the timer/counter logic performing one of timing/counting operations during the programmable hardware element in the instrument executing to perform the measurement function on the signal.

21. The method of claim 1, wherein the programmable hardware element in the instrument executes to perform a process control function using the signal.

22. The method of claim 1, further comprising:

converting the hardware description into a net list; and

compiling the net list format into a hardware program file;

wherein said configuring the programmable hardware element includes downloading the hardware program file to the programmable hardware element to configure the programmable hardware element.

23. The method of claim 22, wherein said converting the hardware description into a net list includes:

utilizing at least one function block from a library of pre-compiled function blocks; and

utilizing hardware target specific information.

24. The method of claim 1, wherein said creating the graphical program includes:

arranging on the screen a plurality of nodes comprising the graphical program and interconnecting the plurality of nodes;

creating and storing data structures which represent the graphical program in response to said arranging and said interconnecting;

wherein said generating the hardware description comprises:

traversing the data structures;

converting the data structures into a hardware description format in response to said traversing.

25. The method of claim 1, wherein the graphical program includes a plurality of nodes;

wherein said generating the hardware description comprises converting each of said nodes into a hardware description format.

26. The method of claim 25, wherein each of said nodes is converted into a hardware description format including an enable input, a clock signal input, and an enable output;

wherein, for a respective node, said enable input receives an enable signal generated from enable out signals from one or more nodes which provide inputs to the respective node.

27. The method of claim 25, wherein the graphical program includes an input terminal;

wherein, for said input terminal, said converting comprises:

determining if data input to the input terminal is from a user interface portion executing on the computer system;

creating a hardware description of a write register, wherein the write register includes one or more data outputs and at least control output.

28. The method of claim 25, wherein the graphical program includes a function node;

wherein, for said function node, said converting comprises:

determining inputs and outputs to/from the function node;

generating a hardware description of logic which performs the function indicated by the function node;

traversing input dependencies of the node;

creating a hardware description of an AND gate, including listing connections of said input dependencies of the node to said AND gate.

29. The method of claim 25, wherein the graphical program includes a structure node;

wherein, for said structure node, said converting comprises:

determining inputs and outputs to/from the structure node;

generating a hardware description of a control block which performs the control function indicated by the structure node;

traversing input dependencies of the node;

creating a hardware description of an AND gate, including listing connections of said input dependencies of the node to said AND gate.

30. The method of claim 25, wherein the graphical program includes an output terminal;

wherein, for said output terminal, said converting comprises:

determining if data output from the output terminal is to a user interface portion executing on the computer system;

creating a hardware description of a read register, wherein the read register includes one or more data inputs and at least control input.

31. The method of claim 1, wherein the graphical program comprises a data flow diagram.

32. The method of claim 1, wherein a first portion of the graphical program is converted into a hardware description;

the method further comprising:

compiling a second portion of the graphical program into machine code for execution by the processor.

33. The method of claim 32, further comprising:

executing the machine code to perform functionality indicated by the second portion of the graphical program;

the programmable hardware element performing functionality indicated by the first portion of the graphical program;

wherein said executing the machine code and the programmable hardware element performing functionality operate to perform functionality indicated by the graphical program.

34. The method of claim 1, wherein the instrument includes a non-volatile memory coupled to the programmable hardware element, the method further comprising:

storing the hardware description into the non-volatile memory;

wherein said configuring the programmable hardware element comprises transferring the hardware description from the non-volatile memory to the programmable hardware element to produce the programmable hardware element.

35. A computer-implemented method for configuring an instrument to perform a measurement function, wherein the instrument includes a programmable hardware element, the method comprising:

creating a graphical program, wherein the graphical program implements the measurement function, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;

generating a hardware description based on the graphical program, wherein the hardware description describes a hardware implementation of the graphical program;

configuring the programmable hardware element in the instrument utilizing the hardware description, wherein after said configuring the programmable hardware element implements a hardware implementation of the graphical program;

wherein the programmable hardware element in the instrument is operable to perform the measurement function on a received signal.

36. The method of claim 35,

wherein the instrument is operable to acquire a signal from an external source after said configuring;

wherein the programmable hardware element in the instrument is operable to perform the measurement function on the signal.

37. The method of claim 35,

wherein the instrument comprises an image acquisition device coupled to a camera;

wherein the camera is operable to acquire an image of an object, and wherein the programmable hardware element in the image acquisition device is operable to perform an image processing function on the image.

38. The method of claim 35,

wherein the instrument comprises a smart camera;

wherein the smart camera is operable to acquire an image of an object and store the image; and

wherein the programmable hardware element in the smart camera is operable to perform an image processing function on the image.

39. The method of claim 35,

wherein the instrument comprises a smart sensor;

wherein the smart sensor is operable to acquire a signal from an external source; and

wherein the programmable hardware element in the smart sensor is operable to perform the measurement function on the signal.

40. The method of claim 35, wherein the smart sensor is one of a temperature sensor, pressure sensor, camera, microphone, flow sensor, position sensor, velocity sensor, acceleration sensor, vibration sensor, electromagnetic sensor, optical sensor, radiation sensor, or environment sensor.

41. The method of claim 35,

wherein the instrument comprises a fieldbus device;

wherein the fieldbus device is operable to acquire the signal from the external source; and

wherein the programmable hardware element in the fieldbus device is operable to perform the measurement function on the signal.

42. The method of claim 35,

wherein the instrument comprises a programmable logic controller (PLC);

wherein the PLC is operable to acquire the signal from the external source; and

wherein the programmable hardware element in the PLC is operable to perform the measurement function on the signal.

43. The method of claim 35,

wherein the instrument comprises a computer-based instrument, wherein the computer-based instrument is configured as a card inserted into a slot of a computer system;

wherein the computer-based instrument is operable to acquire the signal from an external source; and

wherein the programmable hardware element in the computer-based instrument is operable to perform the measurement function on the signal.

44. The method of claim 35,

wherein the instrument comprises a reconfigurable measurement device, wherein the reconfigurable measurement device is configured as a card inserted into a slot of a computer system;

wherein the reconfigurable measurement device is operable to acquire the signal from the external source; and

wherein the programmable hardware element in the reconfigurable measurement device is operable to perform the measurement function on the signal.

45. A measurement system, comprising:

a computer system comprising a processor, memory and a display;

wherein the memory stores a graphical program, wherein the graphical program implements a measurement function, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;

wherein the memory also stores a software program which is executable to generate a hardware description based on the graphical program, wherein the hardware description describes a hardware implementation of the graphical program; and

an instrument coupled to the computer system, wherein the instrument includes:

an input for acquiring a signal from an external source; and

a programmable hardware element coupled to the input, wherein the programmable hardware element in the instrument is configurable utilizing the hardware description, wherein after being configured the programmable hardware element implements a hardware implementation of the graphical program, wherein the programmable hardware element in the instrument is executable to perform the measurement function on an acquired signal.

46. The measurement system of claim 45,

wherein the instrument is operable to acquire a signal from an external source after being configured;

wherein the programmable hardware element in the instrument is operable to perform the measurement function on the signal.

47. The measurement system of claim 45,

wherein the instrument comprises an image acquisition device coupled to a camera;

wherein the camera is operable to acquire an image of an object, and wherein the programmable hardware element in the image acquisition device is operable to perform an image processing function on the image.

48. The measurement system of claim 45,

wherein the instrument comprises a smart camera;

wherein the smart camera is operable to acquire an image of an object and store the image; and

wherein the programmable hardware element in the smart camera is operable to perform an image processing function on the image.

49. The measurement system of claim 45,

wherein the instrument comprises a smart sensor;

wherein the smart sensor is operable to acquire a signal from an external source; and

wherein the programmable hardware element in the smart sensor is operable to perform the measurement function on the signal.

50. The measurement system of claim 45, wherein the smart sensor is one of a temperature sensor, pressure sensor, camera, microphone, flow sensor, position sensor, velocity sensor, acceleration sensor, vibration sensor, electromagnetic sensor, optical sensor, radiation sensor, or environment sensor.

51. The measurement system of claim 45,

wherein the instrument comprises a fieldbus device;

wherein the fieldbus device is operable to acquire the signal from the external source; and

wherein the programmable hardware element in the fieldbus device is operable to perform the measurement function on the signal.

52. The measurement system of claim 45,

wherein the instrument comprises a programmable logic controller (PLC);

wherein the PLC is operable to acquire the signal from the external source; and

wherein the programmable hardware element in the PLC is operable to perform the measurement function on the signal.

53. The measurement system of claim 45,

wherein the instrument comprises a computer-based instrument, wherein the computer-based instrument is configured as a card inserted into a slot of a computer system;

wherein the computer-based instrument is operable to acquire the signal from an external source; and

wherein the programmable hardware element in the computer-based instrument is operable to perform the measurement function on the signal.

54. The measurement system of claim 45,

wherein the instrument comprises a reconfigurable measurement device, wherein the reconfigurable measurement device is configured as a card inserted into a slot of a computer system;

wherein the reconfigurable measurement device is operable to acquire the signal from the external source; and

wherein the programmable hardware element in the reconfigurable measurement device is operable to perform the measurement function on the signal.

55. The measurement system of claim 45,

wherein the graphical program includes a user interface portion;

wherein the user interface portion is operable to be presented on a display during the programmable hardware element in the instrument executing to perform the measurement function on the signal.

56. The measurement system of claim 55,

wherein the user interface portion operates as a front panel for the instrument;

57. The measurement system of claim 55,

wherein the user interface portion is operable to receive user input to control the instrument while the programmable hardware element in the instrument executes to perform the measurement function on the signal.

58. The measurement system of claim 55,

wherein the user interface portion includes one or more user interface objects;

wherein, in generating a hardware description based on the graphical program, the software program is executable to incorporate a register in the hardware description for at least one of the user interface objects;

wherein the programmable hardware element in the instrument is operable to access a register on the programmable hardware element to affect one of said user interface objects.

59. The measurement system of claim 55,

wherein the user interface portion is operable to be compiled into executable code and stored in the memory for execution by the processor;

wherein the processor is operable to execute the executable code from the memory to present the user interface portion on the display.

60. The measurement system of claim 55, wherein the instrument includes a second processor and a second memory;

wherein the user interface portion is operable to be compiled into executable code and stored in the second memory in the instrument for execution by the second processor in the instrument;

wherein the second processor in the instrument executes the executable code from the memory to present the user interface portion on the display during the programmable hardware element in the instrument executing to perform the measurement function on the signal.

61. The measurement system of claim 45, wherein the instrument further comprises:

analog to digital conversion logic coupled to the input and to the programmable hardware element for performing analog to digital conversion logic on an acquired analog signal to produce a digital signal.

62. The measurement system of claim 45, wherein the instrument further includes timer/counter logic;

wherein the timer/counter logic performs one of timing/counting operations while the programmable hardware element in the instrument executes to perform the measurement function on the signal.

63. The measurement system of claim 45, wherein the programmable hardware element in the instrument executes to perform a process control function using the signal.

64. The measurement system of claim 45, wherein the software program stored in the memory of the computer system is further operable to convert the hardware description into a net list;

wherein the computer system is operable to configure the programmable hardware element utilizing the net list.

65. The measurement system of claim 64, wherein the software program stored in the memory of the computer system is further operable to compile the net list format into a hardware program file; and

wherein the computer system is operable to download the hardware program file to the programmable hardware element to configure the programmable hardware element.

66. The measurement system of claim 45, wherein the programmable hardware element comprises a field programmable gate array (FPGA).

67. The measurement system of claim 45, wherein the computer system includes a bus and also includes one or more expansion slots coupled to the bus adapted for receiving expansion cards;

wherein the instrument comprises an expansion card inserted into an expansion slot of the bus.

68. The measurement system of claim 45, wherein the instrument is an external instrument coupled to the computer system.

69. The measurement system of claim 45, wherein the memory of the computer system stores a graphical programming system for creation of the graphical program;

wherein the graphical programming system is executable to arrange on the screen a plurality of nodes comprising the graphical program and interconnect the plurality of nodes in response to user input;

wherein the graphical programming system is further executable to create and store data structures which represent the graphical program in response to said arranging and said interconnecting;

wherein the software program is executable to traverse the data structures and convert the data structures into a hardware description format in response to said traversing.

70. The measurement system of claim 45, wherein a first portion of the graphical program is converted into a hardware description;

wherein the computer system is operable to compile a second portion of the graphical program into machine code for execution by the CPU.

71. The measurement system of claim 70,

wherein the programmable hardware element is operable to perform functionality indicated by the first portion of the graphical program;

wherein the computer system is operable to execute the machine code to perform functionality indicated by the second portion of the graphical program;

wherein said executing the machine code and the programmable hardware element performing functionality operate to perform functionality indicated by the graphical program.

72. The measurement system of claim 45, wherein the instrument includes a non-volatile memory coupled to the programmable hardware element;

wherein the non-volatile memory is operable to store the hardware description;

wherein the non-volatile memory is operable to transfer the hardware description to the programmable hardware element to produce the programmable hardware element.

73. The measurement system of claim 45, wherein the instrument performs data acquisition/generation functions.

74. The measurement system of claim 45, wherein the instrument is a GPIB instrument.

75. The measurement system of claim 45, wherein the instrument is a VXI instrument.

76. The measurement system of claim 45, wherein the instrument is a serial instrument.

77. A measurement system, comprising:

a computer system comprising a processor, memory and a display;

wherein the memory stores a graphical program, wherein the graphical program implements a measurement function, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program, wherein the graphical program also comprises a user interface portion;

wherein the memory also stores a software program which is executable to generate a hardware description based on the graphical program, wherein the hardware description describes a hardware implementation of the graphical program; and

an instrument coupled to the computer system, wherein the instrument includes:

an input for acquiring a signal from an external source; and

a programmable hardware element coupled to the input, wherein the programmable hardware element in the instrument is configurable utilizing the hardware description, wherein after being configured the programmable hardware element implements a hardware implementation of the graphical program, wherein the programmable hardware element in the instrument is executable to perform the measurement function on an acquired signal;

wherein the user interface portion of the graphical program is operable to be presented on a display during the programmable hardware element in the instrument executing to perform the measurement function on the signal.

78. The measurement system of claim 77,

wherein the user interface portion is operable to receive user input to control the instrument while the programmable hardware element in the instrument executes to perform the measurement function on the signal.

79. The measurement system of claim 77,

wherein the user interface portion is operable to be compiled into executable code and stored in the memory of the computer system for execution by the processor of the computer system;

wherein the processor is operable to execute the executable code from the memory to present the user interface portion on the display.

80. The measurement system of claim 77,

wherein the instrument includes a second processor and a second memory comprised in the instrument;

wherein the user interface portion of the graphical program is operable to be compiled into executable code and stored in the second memory of the instrument;

wherein the second processor in the instrument is operable to execute the executable code to present the user interface on the display while the programmable hardware element in the instrument executes to perform the measurement function on the signal.

81. The measurement system of claim 77,

wherein the instrument comprises an image acquisition device coupled to a camera;

wherein the camera is operable to acquire an image of an object, and wherein the programmable hardware element in the image acquisition device is operable to perform an image processing function on the image.

82. The measurement system of claim 77,

wherein the instrument comprises a smart camera;

wherein the smart camera is operable to acquire an image of an object and store the image; and

wherein the programmable hardware element in the smart camera is operable to perform an image processing function on the image.

83. The measurement system of claim 77,

wherein the instrument comprises a smart sensor;

wherein the smart sensor is operable to acquire a signal from an external source; and

wherein the programmable hardware element in the smart sensor is operable to perform the measurement function on the signal.

84. A measurement system, comprising:

a computer system comprising a processor, memory and a display;

wherein the memory stores a graphical program, wherein the graphical program implements a measurement function, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program, wherein the graphical program also comprises a user interface portion;

wherein the memory also stores a software program which is executable to generate a hardware description based on the graphical program, wherein the hardware description describes a hardware implementation of the graphical program; and

an instrument coupled to the computer system, wherein the instrument includes:

an input for acquiring a signal from an external source;

a processor;

a memory coupled to the processor and the input; and

a programmable hardware element coupled to the input, wherein the programmable hardware element in the instrument is configurable utilizing the hardware description, wherein after being configured the programmable hardware element implements a hardware implementation of the graphical program, wherein the programmable hardware element in the instrument is executable to perform the measurement function on an acquired signal;

wherein the user interface portion of the graphical program is operable to be compiled into executable code and stored in the memory of the instrument;

wherein the processor in the instrument is operable to execute the executable code to present the user interface on the display while the programmable hardware element in the instrument executes to pergorm the measurement function on the signal.

85. The measurement system of claim 84,

wherein the instrument comprises an image acquisition device coupled to a camera;

wherein the camera is operable to acquire an image of an object, and wherein the programmable hardware element in the image acquisition device is operabl to perform an image processing function on the image.

86. The measurement system of claim 84,

wherein the instrument comprises a smart camera;

wherein the smart camera is operable to acquire an image of an object and store the image; and

wherein the programmable hardware element in the smart camera is operable to perform an image processing function on the image.

87. The measurement system of claim 84,

wherein the instrument comprises a smart sensor;

wherein the smart sensor is operanle to acquire a signal from an external source; and

wherein the programmable hardware element in the smart sensor is operable to perform the measurement function on the signal.

Description

FIELD OF THE INVENTION

The present invention relates to graphical programming, and in particular to a system and method for converting a graphical program into a hardware implementation. The present invention further relates to a system and method for configuring an instrument or device, such as an image acquisition or machine vision device, a motion control device, or a programmable logic controller, to perform measurement functions, wherein the instrument or device includes a programmable hardware element

DESCRIPTION OF THE RELATED ART

Traditionally, high level text-based programming languages have been used by programmers in writing applications programs. Many different high level programming languages exist, including BASIC, C, FORTRAN, Pascal, COBOL, ADA, APL, etc. Programs written in these high level languages are translated to the machine language level by translators known as compilers or interpreters. The high level programming languages in this level, as well as the assembly language level, are referred to as text-based programming environments.

Increasingly computers are required to be used and programmed by those who are not highly trained in computer programming techniques. When traditional text-based programming environments are used, the user's programming skills and ability to interact with the computer system often become a limiting factor in the achievement of optimal utilization of the computer system.

There are numerous subtle complexities which a user must master before he can efficiently program a computer system in a text-based environment. The task of programming a computer system to model or implement a process often is further complicated by the fact that a sequence of mathematical formulas, mathematical steps or other procedures customarily used to conceptually model a process often does not closely correspond to the traditional text-based programming techniques used to program a computer system to model such a process. In other words, the requirement that a user program in a text-based programming environment places a level of abstraction between the user's conceptualization of the solution and the implementation of a method that accomplishes this solution in a computer program. Thus, a user often must substantially master different skills in order to both conceptually model a system and then to program a computer to model that system. Since a user often is not fully proficient in techniques for programming a computer system in a text-based environment to implement his model, the efficiency with which the computer system can be utilized to perform such modeling often is reduced.

Examples of fields in which computer systems are employed to interact with physical systems are the fields of instrumentation, process control, industrial automation, and simulation. Computer measurement and control of devices such as instruments or industrial automation hardware has become increasingly desirable in view of the increasing complexity and variety of instruments and devices available for use. However, due to the wide variety of possible testing and control situations and environments, and also the wide array of instruments or devices available, it is often necessary for a user to develop a custom program to control a desired system.

As discussed above, computer programs used to control such systems traditionally had to be written in text-based programming languages such as, for example, assembly language, C, FORTRAN, BASIC, etc. Traditional users of these systems, however, often were not highly trained in programming techniques and, in addition, text-based programming languages were not sufficiently intuitive to allow users to use these languages without training. Therefore, implementation of such systems frequently required the involvement of a programmer to write software for control and analysis of instrumentation or industrial automation data. Thus, development and maintenance of the software elements in these systems often proved to be difficult.

U.S. Pat. Nos. 4,901,221; 4,914,568; 5,291,587; 5,301,301; and 5,301,336; among others, to Kodosky et al disclose a graphical system and method for modeling a process, i.e., a graphical programming environment which enables a user to easily and intuitively model a process. The graphical programming environment disclosed in Kodosky et al can be considered a higher and more intuitive way in which to interact with a computer. A graphically based programming environment can be represented at a level above text-based high level programming languages such as C, Basic, Java, etc.

The method disclosed in Kodosky et al allows a user to construct a diagram using a block diagram editor. The block diagram may include a plurality of interconnected icons such that the diagram created graphically displays a procedure or method for accomplishing a certain result, such as manipulating one or more input variables and/or producing one or more output variables. In response to the user constructing a diagram or graphical program using the block diagram editor, data structures may be automatically constructed which characterize an execution procedure which corresponds to the displayed procedure. The graphical program may be compiled or interpreted by a computer.

Therefore, Kodosky et al teaches a graphical programming environment wherein a user places or manipulates icons and interconnects or "wires up" the icons in a block diagram using a block diagram editor to create a graphical "program." A graphical program for performing an instrumentation, measurement or automation function, such as measuring a Unit Under Test (UUT) or device, controlling or modeling instruments, controlling or measuring a system or process, or for modeling or simulating devices, may be referred to as a virtual instrument (VI). Thus, a user can create a computer program solely by using a graphically based programming environment. This graphically based programming environment may be used for creating virtual instrumentation systems, modeling processes, control, simulation, and numerical analysis, as well as for any type of general programming.

A graphical program may have a graphical user interface. For example, in creating a graphical program, a user may create a front panel or user interface panel. The front panel may include various graphical user interface elements or front panel objects, such as user interface controls and/or indicators, that represent or display the respective input and output that will be used by the graphical program or VI, and may include other icons which represent devices being controlled. The front panel may be comprised in a single window of user interface elements, or may comprise a plurality of individual windows each having one or more user interface elements, wherein the individual windows may optionally be tiled together. When the controls and indicators are created in the front panel, corresponding icons or terminals may be automatically created in the block diagram by the block diagram editor. Alternatively, the user can place terminal icons in the block diagram which may cause the display of corresponding front panel objects in the front panel, either at edit time or later at run time. As another example, the front panel may comprise front panel objects, e.g., the GUI, embedded in the block diagram.

During creation of the block diagram portion of the graphical program, the user may select various function nodes or icons that accomplish his desired result and connect the function nodes together. For example, the function nodes may be connected in one or more of a data flow, control flow, and/or execution flow format. The function nodes may also be connected in a "signal flow" format, which is a subset of data flow. The function nodes may be connected between the terminals of the various user interface elements, e.g., between the respective controls and indicators. Thus the user may create or assemble a graphical program, referred to as a block diagram, graphically representing the desired process. The assembled graphical program may be represented in the memory of the computer system as data structures and/or program instructions. The assembled graphical program, i.e., these data structures, may then be compiled or interpreted to produce machine language that accomplishes the desired method or process as shown in the block diagram.

Input data to a graphical program may be received from any of various sources, such as from a device, unit under test, a process being measured or controlled, another computer program, or from a file. Also, a user may input data to a graphical program or virtual instrument using a graphical user interface, e.g., a front panel as described above. The input data may propagate through the data flow block diagram or graphical program and appear as changes on the output indicators. In an instrumentation application, the front panel can be analogized to the front panel of an instrument. In an industrial automation application the front panel can be analogized to the MMI (Man Machine Interface) of a device. The user may adjust the controls on the front panel to affect the input and view the output on the respective indicators. Alternatively, the front panel may be used merely to view the input and output, or just the output, and the input may not be interactively manipulable by the user during program execution.

Thus, graphical programming has become a powerful tool available to programmers. Graphical programming environments such as the National Instruments LabVIEW product have become very popular. Tools such as LabVIEW have greatly increased the productivity of programmers, and increasing numbers of programmers are using graphical programming environments to develop their software applications. In particular, graphical programming tools are being used for test and measurement, data acquisition, process control, man machine interface (MMI), supervisory control and data acquisition (SCADA) applications, simulation, image processing/machine vision applications, and motion control, among others.

A primary goal of graphical programming, including virtual instrumentation, is to provide the user the maximum amount of flexibility to create his/her own applications and/or define his/her own instrument functionality. In this regard, it is desirable to extend the level at which the user is able to program a device, e.g., extend the level at which a user of instrumentation or industrial automation hardware is able to program an instrument. The evolution of the levels at which the user has been able to program an instrument is essentially as follows.

1. User level software (LabVIEW, LabWindows CVI, Visual Basic, etc.)

2. Kernel level software

3. Auxiliary kernel level software (a second kernel running along side the main OS, e.g., InTime, VentureCom, etc.)

4. Embedded kernel level software

5. Hardware level software (FPGA--the present patent application)

In general, going down the above list, the user is able to create software applications which provide a more deterministic real-time response. Currently, most programming development tools for instrumentation or industrial automation provide an interface at level 1 above. In general, most users are unable and/or not allowed to program at the kernel level or auxiliary kernel level. The user level software typically takes the form of software tools that can be used to create software which operates at levels 1 and/or 4.

Current instrumentation solutions at level 5 primarily exist as vendor-defined solutions, i.e., vendor created modules. However, it would be highly desirable to provide the user with the ability to develop user level software which operates at the hardware level. More particularly, it would be desirable to provide the user with the ability to develop high level software, such as graphical programs, which can then be readily converted into hardware level functionality. This would provide the user with the dual benefits of being able to program device functionality at the highest level possible (text-based or graphical programs), while also providing the ability to have the created program operate directly in hardware for increased speed and efficiency.

SUMMARY OF THE INVENTION

One embodiment of the present invention comprises a computer-implemented system and method for automatically generating hardware level functionality, e.g., configured programmable hardware such as FPGAs or CPLDs, in response to a graphical program created by a user. This provides the user the ability to develop or define desired functionality using graphical programming techniques, while enabling the resulting program to operate directly in hardware. The computer-implemented system and method may be used for configuring an instrument to perform a measurement function, wherein the instrument includes the programmable hardware element.

The user may first create a graphical program which performs or represents the desired functionality. The graphical program may comprise a block diagram which includes a plurality of interconnected nodes which visually indicate functionality of the graphical program. The plurality of nodes may be interconnected in one or more of a data flow, control flow, or execution flow format. The graphical program may include a single diagram or a hierarchy of subprograms or sub-diagrams. In one embodiment, the user may place various constructs in portions of the graphical program to aid in conversion of these portions into hardware form. As the user creates or assembles the graphical program on the display, data structures and/or software code may be automatically created and stored in memory corresponding to the graphical program being created. In a measurement or instrumentation application, the graphical program may implement a measurement function.

The user may then select an option to convert the graphical program into executable form, wherein at least a portion of the graphical program is converted into a hardware implementation. In one embodiment, the block diagram portion of the graphical program is converted into a hardware configuration that may be downloaded onto the programmable hardware element comprised on the instrument or measurement device. If the graphical program includes a user interface portion, then the user interface portion may not be converted into a hardware configuration, but rather may be compiled for execution by a processor.

The user may optionally select which portions of a graphical program, e.g., which diagrams (or sub-VIs), are to be translated into hardware form, either during creation of the graphical program or when selecting the option to convert the graphical program into executable form. Thus the user can select a first portion of the graphical program, e.g., a first one or more diagrams which require real time execution, to be downloaded onto the programmable hardware element, and may select other diagrams which do not require real time execution to be executed by a processor. Alternatively, the selection of portions of the graphical program to be compiled for execution by the CPU or to be provided for hardware implementation may be automatically performed by the system.

The portion of the graphical program selected for hardware implementation may first be converted to a an abstract hardware graph, also referred to as a VDiagram tree. More specifically, the data structures corresponding to the graphical program may be converted into the abstract hardware graph or VDiagram tree. Thus in one embodiment the conversion may not be a one-step process of generating a hardware description directly from graphical program internal data structures, but rather may involve the VDiagram intermediate format. The VDiagram tree may comprise data structures representing the functional objects of the graphical program and the data flow, control flow or execution flow between them. VDiagrams are described in detail below.

The VDiagram tree may be parsed by a back end program module which generates a specific hardware description from the tree, such as a VHDL or EDIF hardware description. The hardware description may then converted into a hardware netlist. Netlists for various types of hardware devices may be created from a hardware description (e.g., FPGA-specific netlists, CPLD-specific netlists, etc.). As used herein, the term "netlist" comprises various intermediate hardware-specific description formats comprising information regarding the particular hardware elements required to implement a hardware design and the relationship among those elements. For example, the back end may generate a VHDL file containing a hardware description of the graphical program. An FPGA-specific netlist may then be created from the VHDL file, wherein the netlist comprises information regarding the various FPGA components (and the logical relationship among those components) necessary to implement the hardware design described in the VHDL file.

The process of converting a hardware description such as a VHDL file into a netlist may be performed by readily available synthesis tools, as is well known in the art. The netlist may then be compiled into a hardware program file (also called a software bit stream) which may be used to program a programmable logic device (PLD) or programmable hardware element such as an FPGA or a CPLD, or other types of (re)configurable hardware devices. In one embodiment, the hardware description is converted into an FPGA program file.

The step of compiling the resulting netlist into a PLD program file may use a library of pre-compiled function blocks to aid in the compilation, as well as hardware target specific information. The library of pre-compiled function blocks may include netlist libraries for programmatic structures, such as for/next loops, while/do loops, case structures, and sequence structures, among others. This allows the user to program with high-level programming constructs, such as iteration, looping, and case structures, while allowing the resulting program to execute directly in hardware.

The resulting bit stream may then be transferred to a PLD or other (re)configurable hardware device such as an FPGA to produce a programmed hardware device equivalent to the graphical program or block diagram.

One embodiment of the invention comprises a general purpose computer system which includes a CPU and memory, and an interface card or device coupled to the computer system which includes a programmable hardware element or logic, such as an FPGA. The computer system may include a graphical programming system which is used to develop the graphical program. The computer system may also include software which is operable to convert the graphical program into an abstract hardware graph, and then convert the abstract hardware graph into a hardware description. The computer system further may include a synthesis tool which is used to compile the hardware description into a netlist, as well as other tools for converting the netlist into a PLD program file for uploading into the PLD. The computer system may further include a library of pre-compiled function blocks which are used by the synthesis tool to aid in creating the netlist.

As described above, in one embodiment, a user creates a graphical program and then the system and method of the present invention may operate to convert at least a portion of the graphical program into a hardware description. The hardware description may then be converted to a hardware program which executes on a PLD. The present system and method thus extends the traditional model of software development to include the possibility of running at least a portion (or all) of a graphical program on hardware, instead of running the entire graphical program on the general-purpose processor.

However, it is noted that the use of the present invention is not limited to the creation of application programs as described above. One embodiment comprises a system and method to create an abstract hardware graph from a graphical program. Various back end programs may be called to generate disparate types of hardware descriptions from the abstract hardware graph. These hardware descriptions may be used for purposes other than creating a bit stream for programming a PLD. For example, a hardware description may be used in the process of creating and printing a traditional circuit board which may be produced in mass quantities.

It is also noted that various back end programs may be called to generate software source code, such as C language code, from the abstract hardware graph. As described in detail below, the abstract hardware graph (VDiagram tree) generated by the present method may contain information regarding the execution order and data flow of the graphical program. This information may be used to establish a procedural order in a traditional programming language. Also, since the execution order for portions of a graphical program may be inherently parallel, the system and method described herein are well-suited for creating a program in a text-based parallel programming language.

In one embodiment, during or after the user creates the graphical program, the method may then estimate and optionally display one or more of the size and cost of a hardware implementation of the graphical program. In one embodiment, for example where the graphical program implements a measurement function, the graphical program manipulates one or more hardware resources of the instrument. Examples of hardware resources include A/D converters, D/A converters, timers, counters, clocks, memories, pixel processors, etc. In this embodiment, creating the graphical program includes displaying an indication of usage, or the status of usage, of the one or more hardware resources during creation of the graphical program.

In another embodiment, the user may insert a probe at a location in the graphical program, wherein the probe is operable to display data generated at the location during execution of the graphical program. In this embodiment, the programmable hardware element includes the probe element for implementing probing in the programmable hardware element. It is noted that non real-time clock cycle observations are possible in current FPGA's, such as Xilinx FPGAs, without external/explicit hooks. However, explicit probes/probe interfaces on the program of the FPGA allow external visibility and real-time performance.

In one embodiment, the target device including the programmable hardware element or reconfigurable hardware (or PLD) being programmed comprises an instrument or measurement device. The instrument or measurement device may be a "stand-alone" instrument, or a measurement device or card coupled to or comprised in a computer system. Example instruments or measurement devices include a data acquisition device or card, a GPIB interface card or GPIB instrument, a VXI interface card or VXI instrument card, a PXI card or device, an image processing (or machine vision) device or card, a smart camera, a smart sensor, a computer-based instrument device or card such as an oscilloscope or multimeter device, a motion control device or card, a traditional instrument such as a stand-alone oscilloscope or multimeter, or other measurement or sensing device. The target device including the reconfigurable hardware or PLD being programmed may also comprise an automation or control device, such as a programmable logic controller, a fieldbus card or device, a CAN (Controller Area Network) device or card, a distributed data acquisition device or module, or other type of automation or control device. The target device being programmed may comprise an instrument or device connected to the computer, such as through a serial connection. It is noted that the target instrument or device being programmed, which includes a programmable hardware element or other (re)configurable hardware element, can take any of various forms, as desired.

In the above embodiments, the user may create a graphical program or block diagram which implements a measurement function. For example, where the target device is a computer-based reconfigurable instrument, the measurement function may incorporate oscilloscope functionality to perform a desired measurement. Where the target device is am image acquisition device or smart camera, the measurement function may comprise an image processing or machine vision function that processes an image. Where the target device is a motion control device, the function may comprise a motion control algorithm specified by the user to control a motion device to move an object.

The method may then generate a hardware description based on at least a portion of the graphical program, wherein the hardware description describes a hardware implementation of the at least a portion of the graphical program. The programmable hardware element is then configured in the instrument or device utilizing the hardware description, wherein after configuration the programmable hardware element implements a hardware implementation of the at least a portion of the graphical program. After the hardware element has been configured, the system may be executed. During execution, the instrument may acquire a signal from an external source, and the programmable hardware element in the instrument may execute to perform the measurement function on the signal.

As one example, where the instrument is an image acquisition device or smart camera, the programmable hardware element in the image acquisition device or smart camera can be configured to perform an image processing (or machine vision) function. In this example embodiment, a graphical program may first be created, either by the user or automatically from another representation, such as a prototype. The graphical program may implement an image processing or machine vision function. The graphical program may include a user interface portion and a block diagram portion. A hardware description may then be generated based on the block diagram portion of the graphical program, wherein the hardware description describes a hardware implementation of the block diagram portion of the graphical program. The programmable hardware element in the image acquisition device or smart camera may then be configured utilizing the hardware description. After configuration, the programmable hardware element implements a hardware implementation of the block diagram portion of the graphical program.

During execution, a camera associated with the image acquisition device, or comprised in the smart camera, may acquire an image of an object. For example, the image acquisition device or smart camera may be used in a machine vision inspection application, or a manufacturing assembly application. The programmable hardware element in the image acquisition device or smart camera may then receive the acquired image of the object. The programmable hardware element in the image acquisition device or smart camera may then execute to perform the image processing function (or machine vision function) on the image. The results of this image processing may then be used to make a decision in a machine vision inspection application or may be used to perform an operation in a manufacturing assembly application.

During execution of the programmable hardware element in the image acquisition device or smart camera, the user interface portion may be presented on a display. A user or operator may use the displayed user interface portion to view or control the image processing function (or machine vision function).

Thus the method may operate to configure an instrument, measurement device, or control device to perform measurement functions, wherein the instrument includes a programmable hardware element.

As another example, where the instrument or device is a motion control device, the programmable hardware element in the motion control device can be configured to perform a motion function. In this example embodiment, a graphical program may first be created, either by the user or automatically from another representation, such as a prototype. The graphical program may implement a motion function, i.e., the graphical program may describe a motion sequence. The graphical program may include a user interface portion and a block diagram portion. A hardware description may then be generated based on the block diagram portion of the graphical program, wherein the hardware description describes a hardware implementation of the block diagram portion of the graphical program. The programmable hardware element in the motion control device may then be configured utilizing the hardware description. After configuration, the programmable hardware element implements a hardware implementation of the block diagram portion of the graphical program.

The motion control device may be used in a machine vision inspection application to move an object being inspected or to move a camera. The motion control device may also be used in a manufacturing assembly application to move a part or object being assembled, or may be used in a robotics application, among others. During the respective application, the programmable hardware element in the motion control device may then execute to move the motion control device accordingly.

During execution of the programmable hardware element in the motion control device, the user interface portion may be presented on a display. A user or operator may use the displayed user interface portion to view or control the motion control function.

Thus the method may operate to configure an instrument (which includes various types of measurement devices and/or control devices) to perform measurement functions, wherein the instrument includes a programmable hardware element.

BRIEF DESCRIPTION OF THE DRAWINGS

A better understanding of the present invention can be obtained when the following detailed description of the preferred embodiment is considered in conjunction with the following drawings, in which:

FIG. 1A illustrates an instrumentation control system according to one embodiment of the invention;

FIG. 1B illustrates an industrial automation system according to one embodiment of the invention;

FIG. 2 illustrates a computer system coupled to a smart camera according to one embodiment of the invention;

FIGS. 3A and 3B illustrate image processing systems according to embodiments of the invention;

FIG. 4 is a block diagram of the computer system of FIGS. 1A and 1B;

FIGS. 5A, 5B and 5C are block diagrams illustrating an interface card or device configured with a programmable hardware element according to various embodiments of the present invention;

FIG. 5D is a block diagram illustrating a smart camera configured with a programmable hardware element according to an embodiment of the present invention;

FIG. 6 is a flowchart diagram illustrating configuration of an instrument according to one embodiment of the invention;

FIG. 6A is a flowchart diagram illustrating configuration of an image acquisition device or smart camera according to one embodiment of the invention;

FIG. 7 illustrates the conversion of a graphical program to hardware and software implementations;

FIG. 8 illustrates the generation of various types of hardware and software descriptions from a VDiagram tree;

FIG. 9 illustrates the conversion of a graphical program into a hardware description and the use of the hardware description to program an FPGA;

FIG. 10 illustrates the conversion of a graphical program into a software source code description and the compilation and linkage of the source code;

FIG. 11 is a flowchart diagram illustrating operation of one embodiment of the invention, including compiling a first portion of the graphical program into machine language and converting a second portion of the graphical program into a hardware implementation;

FIG. 12 is a more detailed flowchart diagram illustrating creation of a graphical program according to the preferred embodiment;

FIG. 13 is a graphical representation of a VDiagram and the relationship among its elements;

FIG. 14 is an example VDiagram hierarchy illustrating the relationship of a parent VDiagram, child VDiagram, and srnholder for a simple graphical program;

FIG. 15 is a flowchart diagram illustrating the process of building a VDiagram tree for a graphical program;

FIG. 16 is a simple graphical program used as an example of building a VDiagram;

FIG. 17 is a more detailed flowchart diagram illustrating the step adding VDiagram information for each object in a graphical program;

FIG. 18 is a block diagram representing a VDiagram for the example program of FIG. 16;

FIG. 19 is an example graphical program containing a while loop structure;

FIGS. 20A and 20B are partial graphical programs illustrating aspects of the graphical program of FIG. 19;

FIG. 21 is a flowchart diagram illustrating how the flow of data from a parent VDiagram to a child VDiagram is represented;

FIG. 22 is a block diagram illustrating the relationship among structure controllers, srnholders, and child VDiagrams;

FIG. 23 is a block diagram illustrating how a structure controller drives the enable ports for a VDiagram representing a while loop subframe;

FIG. 24 is an example program hierarchy in which the same global variable is read in one subprogram and written in another subprogram;

FIGS. 25 and 26 are block diagrams illustrating the hardware represented by VDiagrams for various subprograms of FIG. 24;

FIGS. 27 and 28 are block diagrams illustrating the ports, components, and connections added to VDiagrams for various subprograms of FIG. 24;

FIG. 29 is a block diagram illustrating the ports, signals, and components that are created to manage a global variable resource with two read and two write access points;

FIG. 30 is a flowchart diagram illustrating operation where the method exports an output terminal into a hardware description;

FIG. 31 is a flowchart diagram illustrating operation where the method exports an input terminal into a hardware description;

FIG. 32 is a flowchart diagram illustrating how a back end program may generate VHDL syntax from a VDiagram tree;

FIGS. 33 and 34 illustrate a simple example of operation of the present invention, wherein FIG. 33 illustrates a simple graphical program and FIG. 34 is a conceptual diagram of the hardware description of the graphical program of FIG. 33;

FIGS. 35-37 illustrate another example of operation of the present invention, wherein FIG. 35 illustrates a graphical program, FIG. 36 illustrates a tree of data structures created in response to the graphical program of FIG. 35, and FIG. 37 is a conceptual diagram of the hardware description of the graphical program of FIG. 35;

FIGS. 38-39 illustrate a graphical program called example1.vi.

While the invention is susceptible to various modifications and alternative forms specific embodiments are shown by way of example in the drawings and will herein be described in detail. It should be understood however, that drawings and detailed description thereto are not intended to limit the invention to the particular form disclosed. But on the contrary the invention is to cover all modifications, equivalents and alternative following within the spirit and scope of the present invention as defined by the appended claims.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Incorporation by Reference

The following U.S. Patents and patent applications are hereby incorporated by reference in their entirety as though fully and completely set forth herein.

U.S. Pat. No. 4,914,568 titled "Graphical System for Modeling a Process and Associated Method," issued on Apr. 3, 1990.

U.S. Pat. No. 5,481,741 titled "Method and Apparatus for Providing Attribute Nodes in a Graphical Data Flow Environment".

U.S. Pat. No. 6,219,628 titled "System and Method for Configuring an Instrument to Perform Measurement Functions Utilizing Conversion of Graphical Programs into Hardware Implementations".

U.S. Pat. No. 6,173,438 titled "Embedded Graphical Programming System" filed Aug. 18, 1997, whose inventors are Jeffrey L. Kodosky, Darshan Shah, Samson DeKey, and Steve Rogers.

U.S. patent application Ser. No. 09/745,023 titled "System and Method for Programmatically Generating a Graphical Program in Response to Program Information," filed Dec. 20, 2000.

U.S. patent application Ser. No. 09/595,003 titled "System and Method for Automatically Generating a Graphical Program to Implement a Prototype", filed Jun. 13, 2000.

U.S. patent application Ser. No. 09/587,682 titled "System and Method for Automatically Generating a Graphical Program to Perform an Image Processing Algorithm", filed on Jun. 5, 2000.

The LabVIEW and BridgeVIEW graphical programming manuals, including the "G Programming Reference Manual", available from National Instruments Corporation, are also hereby incorporated by reference in their entirety.

FIGS. 1A, 1B, 2, 3A and 3B--Instrumentation and Industrial Automation Systems

The following describes embodiments of the present invention involved with performing test and/or measurement functions and/or controlling and/or modeling instrumentation or industrial automation hardware. However, it is noted that the present invention can be used to create hardware implementations of graphical programs for a plethora of applications and are not limited to instrumentation or industrial automation applications. In other words, the following description is exemplary only, and the present invention may be used in any of various types of systems. Thus, the system and method of the present invention is operable for automatically creating hardware implementations of graphical programs or graphical code for any of various types of applications, including the control of other types of devices such as multimedia devices, video devices, audio devices, telephony devices, Internet devices, etc., as well as general purpose software applications such as word processing, spreadsheets, network control, games, etc.

FIG. 1A illustrates an exemplary instrumentation control system 100. The system 100 comprises a host computer 102 which connects to one or more instruments. The host computer 102 may comprise a CPU, a display screen, memory, and one or more input devices such as a mouse or keyboard as shown. The computer 102 may operate with the one or more instruments to analyze, measure or control a unit under test (UUT) or process 150. Alternatively, the computer 102 may be used only to configure a programmable hardware element in one or more of the instruments to perform a measurement or control function. As used herein, the term "programmable hardware element" is intended to include various types of programmable hardware, reconfigurable hardware, programmable logic, or field-programmable devices (FPDs), such as one or more FPGAs (Field Programmable Gate Arrays), or one or more PLDs (Programmable Logic Devices), such as one or more Simple PLDs (SPLDs) or one or more Complex PLDs (CPLDs), or other types of programmable hardware.

The one or more instruments may include a GPIB instrument 112 and associated GPIB interface card 122, a data acquisition board 114 and associated signal conditioning circuitry 124, a VXI instrument 116, a PXI instrument 118, a video device or camera 132 and associated image acquisition (or machine vision) card 134, a motion control device 136 and associated motion control interface card 138, and/or one or more computer based instrument cards 142, among other types of devices.

The GPIB instrument 112 may be coupled to the computer 102 via the GPIB interface card 122 provided by the computer 102. In a similar manner, the video device 132 may be coupled to the computer 102 via the image acquisition card 134, and the motion control device 136 may be coupled to the computer 102 through the motion control interface card 138. The data acquisition board 114 may be coupled to the computer 102, and may interface through signal conditioning circuitry 124 to the UUT. The signal conditioning circuitry 124 may comprise an SCXI (Signal Conditioning eXtensions for Instrumentation) chassis comprising one or more SCXI modules 126.

The GPIB card 122, the image acquisition card 134, the motion control interface card 138, and the DAQ card 114 are typically plugged in to an I/O slot in the computer 102, such as a PCI bus slot, a PC Card slot, or an ISA, EISA or MicroChannel bus slot provided by the computer 102. However, these cards 122, 134, 138 and 114 are shown external to computer 102 for illustrative purposes. These devices may also be connected to the computer 102 through a serial bus or through other means.

The VXI chassis or instrument 116 may be coupled to the computer 102 via a VXI bus, MXI bus, or other serial or parallel bus provided by the computer 102. The computer 102 may include VXI interface logic, such as a VXI, MXI or GPIB interface card (not shown), which interfaces to the VXI chassis 116. The PXI chassis or instrument may be coupled to the computer 102 through the computer's PCI bus.

A serial instrument (not shown) may also be coupled to the computer 102 through a serial port, such as an RS-232 port, USB (Universal Serial bus) or IEEE 1394 or 1394.2 bus, provided by the computer 102. In typical instrumentation control systems an instrument will not be present of each interface type, and in fact many systems may only have one or more instruments of a single interface type, such as only GPIB instruments.

In the embodiment of FIG. 1A, one or more of the devices connected to the computer 102 include a programmable hardware element according to an embodiment of the present invention. For example, one or more of the GPIB card 122, the DAQ card 114, a VXI card in VXI chassis 116, a PXI card in PXI chassis 118, the image acquisition board 134, the motion control board 138, or a computer-based instrument 142, include a programmable hardware element according to an embodiment of the present invention. Alternatively, or in addition, one or more of the GPIB instrument 112, the VXI instrument 116, the serial instrument, or another type of device include a programmable hardware element according to an embodiment of the present invention. The programmable hardware element may be configured to perform a measurement function. Where the programmable hardware element is comprised on a VXI or PXI card, the programmable hardware may also or instead be configured to control one or more other VXI or PXI cards, respectively, comprised in the respective chassis. In one embodiment, the programmable hardware comprises a field programmable gate array (FPGA).

The instruments may be coupled to a unit under test (UUT) or process 150, or may be coupled to receive field signals, typically generated by transducers. The system 100 may be used in a data acquisition and control application, in a test and measurement application, an image processing or machine vision application, a process control application, a man-machine interface application, a simulation application, or a hardware-in-the-loop validation application.

FIG. 1B illustrates an exemplary industrial automation system 160. The industrial automation system 160 is similar to the instrumentation or test and measurement system 100 shown in FIG. 1A. Elements which are similar or identical to elements in FIG. 1A have the same reference numerals for convenience. The system 160 may comprise a computer 102 which connects to one or more devices or instruments. The computer 102 may comprise a CPU, a display screen, memory, and one or more input devices such as a mouse or keyboard as shown. The computer 102 may operate with the one or more devices to a process or device 150 to perform an automation function, such as MMI (Man Machine Interface), SCADA (Supervisory Control and Data Acquisition), portable or distributed data acquisition, process control, advanced analysis, or other control. Alternatively, the computer 102 may be used only to configure a programmable hardware element on one or more of the instruments or devices to perform a measurement, automation or control function.

The one or more devices may include a data acquisition board 114 and associated signal conditioning circuitry 124, a PXI instrument 118, a video device 132 and associated image acquisition card 134, a motion control device 136 and associated motion control interface card 138, a fieldbus device 170 and associated fieldbus interface card 172, a PLC (Programmable Logic Controller) 176, a serial instrument 182 and associated serial interface card 184, or a distributed data acquisition system, such as the Fieldpoint system available from National Instruments, among other types of devices.

The DAQ card 114, the PXI chassis 118, the video device 132, and the image acquisition card 136 may be connected to the computer 102 as described above. The serial instrument 182 may be coupled to the computer 102 through a serial interface card 184, or through a serial port, such as an RS-232 port, provided by the computer 102. The PLC 176 may couple to the computer 102 through a serial port, Ethernet port, or a proprietary interface. The fieldbus interface card 172 may be comprised in the computer 102 and may interface through a fieldbus network to one or more fieldbus devices. Each of the DAQ card 114, the serial card 184, the fieldbus card 172, the image acquisition card 134, and the motion control card 138 are typically plugged in to an I/O slot in the computer 102 as described above. However, these cards 114, 184, 172, 134, and 138 are shown external to computer 102 for illustrative purposes. In typical industrial automation systems a device will not be present of each interface type, and in fact many systems may only have one or more devices of a single interface type, such as only PLCs. The devices may be coupled to the device or process 150.

In the embodiment of FIG. 1B, one or more of the devices connected to the computer 102 may include a programmable hardware element. For example, one or more of the data acquisition board 114, the serial instrument 142, the serial interface card 152, the PLC 144, or the fieldbus network card 156 may include a programmable hardware element according to an embodiment of the present invention. The programmable hardware may comprise a field programmable gate array (FPGA).

As used herein, the term "instrument" is intended to include any of the devices that are adapted to be connected to a computer system as shown in FIGS. 1A and 1B, traditional "stand-alone" instruments, as well as other types of measurement and control devices. The term "measurement function" may include any type of data acquisition, measurement or control function, such as that implemented by the instruments shown in FIGS. 1A and 1B. For example, the term "measurement function" includes acquisition and/or processing of an image. As described below, a graphical program may be created that implements a measurement function, and the programmable hardware element on an instrument may be configured with a hardware implementation of the graphical program. After the programmable hardware element on an instrument has been configured with a hardware implementation of a graphical program, the instrument may be operable to acquire a signal, and perform the measurement function on the acquired signal.

In the embodiments of FIGS. 1A and 1B above, the programmable hardware element may be comprised in a device which connects to the computer 102 over a network, such as the Internet. In one embodiment, the user operates to select a target device from a plurality of possible target devices for programming or configuration according to the present invention. Thus the user may create a graphical program on a computer and configure a programmable hardware element on a target device that is remotely located from the computer and coupled to the computer through a network.

Referring again to FIGS. 1A and 1B, the computer 102 may include a memory medium on which computer programs according to the present invention may be stored. As used herein, the term "memory medium" includes a non-volatile medium, e.g., a magnetic media or hard disk, or optical storage; a volatile medium, such as computer system memory, e.g., random access memory (RAM) such as DRAM, SRAM, EDO RAM, RAMBUS RAM, DR DRAM, etc.; or an installation medium, such as a CD-ROM or floppy disks 104, on which the computer programs according to the present invention may be stored for loading into the computer system. The term "memory medium" may also include other types of memory or combinations thereof.

The memory medium may be comprised in the computer 102 where the programs are executed or may be located on a second computer which is coupled to the computer 102 through a network, such as a local area network (LAN), a wide area network (WAN), or the Internet. In this instance, the second computer operates to provide the program instructions through the network to the computer 102 for execution.

The software programs of the present invention are stored in a memory medium of the respective computer 102, or in a memory medium of another computer, and executed by the CPU. The CPU executing code and data from the memory medium thus comprises a means for converting graphical programs into hardware implementations according to the steps described below.

The memory medium may store a graphical programming development system for developing graphical programs. The memory medium may also store one or more computer programs which are executable to convert at least a portion of a graphical program into a form for configuring or programming the programmable hardware element.

The instruments or devices in FIGS. 1A and 1B are controlled by or configured by graphical software programs, optionally a portion of which execute on the CPU of the computer 102, and at least a portion of which are downloaded to the programmable hardware element on an instrument or device for hardware execution. The graphical software programs which perform data acquisition, analysis and/or presentation, e.g., for measurement, instrumentation control, industrial automation, or simulation, may be referred to as virtual instruments.

In the present application, the term "graphical program" or "block diagram" is intended to include a program comprising graphical code, e.g., two or more nodes or icons interconnected in one or more of a data flow, control flow, or execution flow format, wherein the interconnected nodes or icons may visually indicates the functionality of the program. Thus the terms "graphical program" or "block diagram" are each intended to include a program comprising a plurality of interconnected nodes or icons which visually indicates the functionality of the program. A graphical program may comprise a block diagram and may also include a user interface portion or front panel portion. The user interface portion may be contained in the block diagram or may be contained in one or more separate panels or windows. A graphical program may be created using any of various types of systems which are used to develop or create graphical code or graphical programs, including LabVIEW, DASYLab, and DiaDem from National Instruments, Visual Designer from Intelligent Instrumentation, Agilent VEE (Visual Engineering Environment), Snap-Master by HEM Data Corporation, SoftWIRE from Measurement Computing, ObjectBench by SES (Scientific and Engineering Software), Simulink from the MathWorks, WiT from Coreco, Vision Program Manager from PPT Vision, Hypersignal, VisiDAQ, VisSim, and Khoros, among others. In the preferred embodiment, the system uses the LabVIEW graphical programming system available from National Instruments.

Although in the preferred embodiment the graphical programs and programmable hardware are involved with measurement applications, including data acquisition/generation, analysis, and/or display, and for controlling or modeling instrumentation or industrial automation hardware, as noted above the present invention can be used to create hardware implementations of graphical programs for a plethora of applications and is not limited to measurement, instrumentation or industrial automation applications. In other words, FIGS. 1A and 1B are exemplary only, and the present invention may be used in any of various types of systems. Thus, the system and method is operable for automatically creating hardware implementations of graphical programs or graphical code for any of various types of applications, including general purpose software applications such as word processing, spreadsheets, network control, games, etc.

FIG. 2

FIG. 2 illustrates a computer system 102 coupled to a smart camera 190. The smart camera is an example of an image acquisition device. As used herein, the term "image acquisition device" is intended to include any of various types of devices that are operable to acquire and/or store an image. An image acquisition device may also optionally be further operable to analyze or process the acquired or stored image. Examples of an image acquisition device include the image acquisition (or machine vision) card (also called a video capture board) 134, a device external to a computer that operates similarly to the image acquisition card 134, a smart camera, a robot having machine vision, and other similar types of devices.

As used herein, the terms "image processing" and "machine vision" are used interchangeably to refer to the processing of images to extract useful information from the image or determine characteristics of the image (or to determine characteristics of one or more objects displayed in the image). The term "image processing" is used herein to refer to both "image processing" and "machine vision", to the extent these terms have different meanings. The term "image processing function" includes tools such as edge detection, blob analysis, pattern matching, and other image processing functions. The term "image processing function" may also include an operation or decision that is performed in response to the information extracted or characteristics determined from the image. The term "image processing function" is also intended to include an image processing (or machine vision) algorithm that combines a sequence of two or more image processing functions or tools and/or decision operations that process an image in a desired way or which implement an image processing or machine vision application, such as part inspection, automated assembly, etc.

The computer system 102 shown in FIG. 2 may include a memory medium as described above. As noted above, the memory medium may store a graphical programming development system for developing graphical programs. The graphical programming development system may be used to develop a graphical program that implements an image processing function. In this example, one or more of the nodes in the graphical program may implement or represent an image processing function.

The memory medium may also store one or more computer programs which are executable to convert at least a portion of a graphical program into a form for configuring or programming the programmable hardware or FPGA comprised in the smart camera 190.

The smart camera 190 may include a programmable hardware element (programmable or reconfigurable hardware) according to an embodiment of the present invention, e.g., an FPGA. The programmable hardware element in the smart camera 190 may be configured with a graphical program that implements the image processing function. The smart camera 190 may also comprise a camera coupled to the programmable hardware element. The smart camera 190 may also include a memory (a memory medium) coupled to the camera that stores an acquired image. If the smart camera 190 includes an analog camera, the smart camera 190 may further include analog to digital (A/D) logic for converting analog image signals into a digital image for storage in the memory. The smart camera 190 may also optionally include timer/counter logic that may perform timing/counting operations, e.g., during operation of the programmable hardware element. An embodiment of the smart camera 190 is shown in FIG. 6D.

FIGS. 3A and 3B

FIGS. 3A and 3B illustrate image processing or machine vision systems according to embodiments of the invention. The image processing system of FIG. 3A may comprise a computer 102 and a smart camera 190, and may further include a motion control device 192. The image processing system of FIG. 3B may comprise smart camera 190 and motion control device 190, and may not include computer system 102.

The smart camera 190 may include a digital camera that acquires a digital video signal which comprises an image, or a sequence of images, or other data desired to be acquired. The smart camera 190 may instead include an analog camera that acquires an analog video signal, and the smart camera 190 may further include A/D converters for converting the analog video signal into a digital image.

The smart camera 190 may include a programmable hardware element that has been configured with a hardware implementation corresponding to a graphical program. The programmable hardware element may be configured to perform an image processing function. Thus a graphical program may have been first created that performs the image processing function, and the graphical program may then have been converted into a hardware implementation that is programmed onto the programmable hardware element.

The digital video signal or digital image is provided to the programmable hardware element in the smart camera 190, wherein the image processing function is performed.

In the embodiment of FIG. 3A, the programmable hardware element in the smart camera 190 may perform a portion or all of the image processing function, and the computer 102 may perform a portion of the image processing function. For example, the programmable hardware element in the smart camera 190 may perform the actual processing of the image to determine characteristics of the image, and the computer 102 may then perform an operation based on this result, such as rejecting a part from an assembly line. As another example, the programmable hardware element in the smart camera 190 may perform the processing of the image to determine characteristics of the image, and may also optionally perform an operation based on this result, and the computer system 102 may execute a user interface for the system, e.g., the computer system 102 may execute a user interface portion of a graphical program, wherein the block diagram of this graphical program was used to configure the programmable hardware element in the smart camera 190.

In the embodiment of FIG. 3B, the programmable hardware element in the smart camera 190 may perform all of the desired image processing function, including optionally performing an operation based on determined characteristics of the image, and hence the computer system 102 is not necessary during operation.

In the embodiments of FIGS. 3A and 3B, the programmable hardware element in the smart camera 190 (or the computer system 102 in FIG. 3A) may control the motion control device 192. In an alternate embodiment, the motion control device 192 may also include a programmable hardware element that has been configured with a hardware implementation corresponding to a graphical program. The programmable hardware element may be configured to perform a motion control function. Thus a graphical program may have been first created that performs the motion control function, and the graphical program may then have been converted into a hardware implementation that is programmed onto the programmable hardware element in the motion control device 192. Examples of motion control functions include moving a part or object to be imaged by a camera, rejecting a part on an assembly line, or placing or affixing components on a part being assembled, or a robotics application, among others.

FIG. 4--Computer Block Diagram

FIG. 4 is an exemplary block diagram of the computer 102 of FIGS. 1A, 1B, and 3A. The elements of a computer not necessary to understand the operation of the present invention have been omitted for simplicity. The computer 102 may include at least one central processing unit (CPU) or processor 160 which is coupled to a processor or host bus 162. The CPU 160 may be any of various types, including an x86 processor, a PowerPC processor, a CPU from the Motorola family of processors, a CPU from the SPARC family of RISC processors, as well as others. Main memory 166 is coupled to the host bus 162 by means of memory controller 164. The main memory 166 stores a graphical programming system, and also stores software for converting at least a portion of a graphical program into a hardware implementation. This software will be discussed in more detail below. The main memory 166 may also store operating system software as well as the software for operation of the computer system, as well known to those skilled in the art.

The host bus 162 is coupled to an expansion or input/output bus 170 by means of a bus controller 168 or bus bridge logic. The expansion bus 170 is preferably the PCI (Peripheral Component Interconnect) expansion bus, although other bus types can be used. The expansion bus 170 may include slots for various devices, the examples shown being data acquisition board 114, GPIB interface card 122 which provides a GPIB bus interface to the GPIB instrument 112, and an image acquisition device 134. The computer 102 may further comprises a video display subsystem 180 and hard drive 182 coupled to the expansion bus 170.

One or more of the interface cards or devices (e.g., those shown in FIGS. 1A, 1B and 2) may be coupled to or comprised in the computer 102 and may comprise a programmable hardware element, such as a field programmable gate array (FPGA). The computer 102 may also include a network interface for coupling to a network, wherein the target device or instrument containing the programmable hardware may be coupled to the network. Thus the computer 102 may be used for configuring programmable hardware in a target device over a network.

FIGS. 5A-5D--Programmable Hardware Diagram

FIG. 5A is a block diagram illustrating a device, e.g., an interface card, configured with programmable hardware according to one embodiment. It is noted that FIG. 6A is exemplary only, and an interface card or device configured with programmable hardware according to the present invention may have various architectures or forms, as desired. For example, the device may be internal or external to the computer 102, and may be connected to the computer through a network, such as the Internet. The interface card illustrated in FIG. 5A may be the DAQ interface card 114 shown in either of FIG. 1A, 1B, or 4, or may be the image acquisition device shown in FIG. 1A, 1B, or 4. However, as noted above, the programmable hardware may be included on any of the various devices shown in FIG. 1A or 1B, or on other devices, as desired. Also, the programmable hardware illustrated in FIG. 5A is an FPGA, but the device may include another type of programmable hardware instead, such as a CPLD or other type of (re)configurable hardware. In FIGS. 5A-5C, the interface card is described as being DAQ card 114.

As shown in FIG. 6A, the interface card 114A may include an I/O connector 202 which is coupled for receiving signals. In the embodiments of FIGS. 1A and 1B, the I/O connector 202 presents analog and/or digital connections for receiving/providing analog or digital signals. The I/O connector 202 may be adapted for coupling to SCXI conditioning logic 124 and 126, or may be adapted to be coupled directly to a unit under test 130 or process or system 160.

The interface card 114A may also include data acquisition (DAQ) logic 204. As shown, the data acquisition logic 204 may comprise analog to digital (A/D) converters, digital to analog (D/A) converters, timer counters (TC) and signal conditioning (SC) logic as shown. The DAQ logic 204 may provide the data acquisition functionality of the DAQ card 114. In one embodiment, the DAQ logic 204 comprises 4 A/D converters, 4 D/A converters, 23 digital I/Os, a RTSI connector, and a TIO. This extra hardware is useful for signal processing and motion control applications. The programmable hardware element or FPGA 206 can access these resources directly, thereby enabling creation of very powerful measurement, DSP and control applications, among others.

The interface card 114A includes a programmable hardware element 206. In one embodiment, the programmable hardware 206 comprises a field programmable gate array (FPGA) such as those available from Xilinx, Altera, etc. The programmable hardware element 206 may be coupled to the DAQ logic 204 and may also be coupled to the local bus interface 208. Thus a graphical program can be created on the computer 102, or on another computer in a networked system, and at least a portion of the graphical program can be converted into a hardware implementation form for execution in the FPGA 206. The portion of the graphical program converted into a hardware implementation form is preferably a portion which requires fast and/or real-time execution

In the embodiment of FIG. 5A, the interface card 114A may further include a dedicated on-board processor 212 and memory 214. This enables a portion of the graphical program to be compiled into machine language for storage in the memory 214 and execution by the processor 212. This is in addition to a portion of the graphical program being converted into a hardware implementation form in the FPGA 206. Thus, in one embodiment, after a graphical program has been created, a portion of the graphical program may be compiled for execution on the embedded processor 212 and executes locally on the interface card 114A via the processor 212 and memory 214, and a second portion of the graphical program may be translated or converted into a hardware executable format and uploaded to the FPGA 206 for hardware implementation.

As one example, a first portion of a block diagram (that requires real time or fast execution) of a graphical program may be converted into a hardware executable format and downloaded to the FPGA 206 for hardware implementation, and a second portion of a block diagram (that may not require real time performance) may be stored in the memory 214 as program instructions and executed by the processor 212, in either a compiled or interpreted manner. As another example, a portion or all of the block diagram portion of the graphical program may be converted into a hardware executable format and downloaded to the FPGA 206 for hardware implementation, and a user interface portion (or front panel portion) of the graphical program may be stored in the memory 214 as program instructions and executed by the processor 212, in either a compiled or interpreted manner. Thus the portion of the graphical program which requires the most real time or deterministic performance may be executed directly in hardware for fast operation, and other parts of the block diagram, or the user interface portion, which may not require real time performance, may execute on the processor 212. Where the processor executes the user interface portion, the processor may then send resulting signals to the video subsystem for display of the user interface on the computer display.

As shown, the interface card 114A may further include bus interface logic 216 and a control/data bus 218. In one embodiment, the interface card 114A is a PCI bus-compliant interface card adapted for coupling to the PCI bus of the host computer 102, or adapted for coupling to a PXI (PCI eXtensions for Instrumentation) bus. The bus interface logic 216 and the control/data bus 218 thus present a PCI or PXI interface.

The interface card 114A may also include local bus interface logic 208. In one embodiment, the local bus interface logic 208 may present a RTSI (Real Time System Integration) bus for routing timing and trigger signals between the interface card 114A and one or more other devices or cards.

In one embodiment, the interface card 114A also includes a non-volatile memory (not shown) coupled to the programmable hardware element 206. The non-volatile memory may be operable to store the hardware description received from the host computer system to enable execution of the hardware description in the programmable hardware element 206 prior to or during booting of the computer system 102.

In the embodiment of FIG. 5B, the processor 212 and memory 214 are not included on the interface card 114B, and thus only the portion of the graphical program which is converted into hardware implementation form is uploaded to the FPGA 206. Thus in the embodiment of FIG. 5B, any supervisory control portion of the graphical program which is necessary or desired to execute on a programmable processor in software may be executed by the host CPU in the computer system 102, and is not executed locally by a processor on the interface card 114B.

In the embodiment of FIG. 5C, the processor 212 is not included on the interface card 114C, i.e., the interface card 114C includes the FPGA 206 and the memory 214. In this embodiment, the memory 214 may be used for storing FPGA state information.

FIG. 5D illustrates a block diagram of smart camera 190. As shown, the smart camera 190 may include a programmable hardware element (programmable or reconfigurable hardware) 206 according to an embodiment of the present invention, e.g., an FPGA. The programmable hardware element 206 in the smart camera 190 may be configured with a graphical program that implements an image processing function. The smart camera 190 may also comprise a camera 282 coupled to the programmable hardware element 206. The smart camera 190 may also include a memory (a memory medium) 284 coupled to the camera that stores an acquired image. The memory 284 may be designed to store a portion of an image, a whole image, or two or more images. The memory 284 may include a memory controller (not shown). If the smart camera 190 includes an analog camera, the smart camera 190 may further include analog to digital (A/D) logic (not shown) for converting analog image signals into a digital image for storage in the memory. The smart camera 190 may also optionally include timer/counter logic 286 that may perform timing/counting operations, e.g., during operation of the programmable hardware element. The smart camera 190 may also optionally include a non-volatile memory 288. The smart camera 190 may also include a processor and memory in addition to the programmable hardware element.

FIG. 6--Configuring an Instrument to Perform a Measurement Function

FIG. 6 illustrates a method for configuring an instrument to perform a measurement function. The method shown in FIG. 6 may be used to configure any of the instruments shown in FIGS. 1A and 1B to perform a measurement function. As shown, this method may operate as follows.

First, in step 262 a graphical program may be created on the computer system 102 (or on a different computer system). The graphical program may be created or assembled by the user arranging on a display a plurality of nodes or icons and then interconnecting the nodes to create the graphical program. In response to the user assembling the graphical program, data structures (and/or program instructions) may be created and stored which represent the graphical program. The nodes may be interconnected in one or more of a data flow, control flow, or execution flow format. The graphical program may thus comprise a plurality of interconnected nodes or icons which visually indicate the functionality of the program. As noted above, the graphical program may comprise a block diagram and may also include a user interface portion or front panel portion. Where the graphical program includes a user interface portion, the user may assemble the user interface on the display. As one example, the user may use the LabVIEW graphical programming development environment to create the graphical program.

In an alternate embodiment, the graphical program may be created in step 262 by the user creating or specifying a prototype, followed by automatic or programmatic creation of the graphical program from the prototype. This functionality is described in U.S. patent application Ser. No. 09/595,003 titled "System and Method for Automatically Generating a Graphical Program to Implement a Prototype", incorporated by reference above. The graphical program may be created in other manners, either by the user or programmatically, as desired. The graphical program may implement a measurement function that is desired to be performed by the instrument.

In step 264 the instrument may be coupled to the computer system 102. As noted above, the instrument may comprise a programmable hardware element. It is noted that the instrument or measurement device may be coupled to the computer system 102 before, during or after the graphical program is created.

In step 266 a hardware configuration based on the graphical program may be downloaded onto the programmable hardware element in the instrument to configure the programmable hardware element. The hardware configuration corresponds to a hardware implementation of the graphical program. Downloading the hardware configuration onto the programmable hardware element may comprise the following steps: generating a hardware description based on the graphical program, wherein the hardware description describes a hardware implementation of the graphical program; converting the hardware description into a net list; compiling the net list format into a hardware program file; and downloading the hardware program file to the programmable hardware element to configure the programmable hardware element. These operations are discussed in greater detail below.

After the downloading step is performed, the programmable hardware element is configured with a hardware implementation of the graphical program. At this point, the instrument may be used with the computer system 102, or may be used separately from the computer system 102. If the instrument is a card or board that is designed to be located in a slot of the computer 102, the instrument card or board may optionally remain in the computer system 102 for use, or may be transferred to a different computer system. Alternatively, in step 268 the instrument may be optionally disconnected from the computer system 102, and may possibly be deployed for operation. As another alternative, the method described above may be repetitively used to manufacture a plurality of instruments for later sale. In another embodiment, the method described above may be used to configure an instrument that has already been deployed, wherein the instrument is configured over a network, i.e., where the hardware configuration is downloaded onto the instrument over a network.

After the programmable hardware element in the instrument is configured with a hardware implementation of the graphical program, the instrument can be used in an application. Thus the instrument may acquire a signal in step 270, e.g., from a UUT or system. The programmable hardware element in the instrument may then execute in step 272 to perform the measurement function on the signal.

While the programmable hardware element in the instrument executes to perform the measurement function on the acquired signal, if the graphical program includes a user interface portion, this user interface portion may optionally be presented on a display during this time in step 274. The code corresponding to the user interface portion may be executed by a processor in the computer system 102 or by a processor on the instrument. The user interface portion may operate as a front panel for the instrument. The user may optionally provide user input to the user interface portion on the display to control the instrument while the programmable hardware element in the instrument executes to perform the measurement function on the signal.

FIG. 6A--Configuring an Image Acquisition Device or Smart Camera to Perform an Image Processing Function

FIG. 6A is an exemplary embodiment illustrating a method for configuring an image acquisition device, such as smart camera 190, to perform an image processing function. The method shown in FIG. 6A is an example of the method of FIG. 6 described above, wherein the method is described with reference to an image acquisition device or smart camera. As shown, this method may operate as follows.

First, in step 262A a graphical program may be created on the computer system 102 (or on a different computer system). The graphical program may be created or assembled by the user arranging on a display a plurality of nodes or icons and then interconnecting the nodes to create the graphical program. In response to the user assembling the graphical program, data structures (and/or program instructions) may be created and stored which represent the graphical program. The nodes may be interconnected in one or more of a data flow, control flow, or execution flow format. The graphical program may thus comprise a plurality of interconnected nodes or icons which visually indicates the functionality of the program. As noted above, the graphical program may comprise a block diagram and may also include a user interface portion or front panel portion. Where the graphical program includes a user interface portion, the user may assemble the user interface on the display. As one example, the user may use the LabVIEW graphical programming development environment to create the graphical program.

In an alternate embodiment, the graphical program may be created in step 262 by the user creating or specifying a prototype, followed by automatic or programmatic creation of the graphical program from the prototype. This functionality is described in U.S. patent application Ser. No. 09/587,682 titled "System and Method for Automatically Generating a Graphical Program to Perform an Image Processing Algorithm", incorporated by reference above. The graphical program may be created in other manners, either by the user or programmatically, as desired. In the present example where the instrument is an image acquisition device (e.g., smart camera 190), the graphical program may implement an image processing function.

In step 264 the image acquisition device (e.g., smart camera 190) may be coupled to the computer system 102. As noted above, the smart camera 190 may comprise a camera and a programmable hardware element. It is noted that the image acquisition device may be connected to the computer system 102 before, during or after the graphical program is created.

In step 266 a hardware configuration based on the graphical program may be downloaded onto the programmable hardware element in the image acquisition device (e.g., smart camera 190) to configure the programmable hardware element. The hardware configuration corresponds to a hardware implementation of the graphical program. Downloading the hardware configuration onto the programmable hardware element may comprise the following steps: generating a hardware description based on the graphical program, wherein the hardware description describes a hardware implementation of the graphical program; converting the hardware description into a net list; compiling the net list format into a hardware program file; and downloading the hardware program file to the programmable hardware element to configure the programmable hardware element. These steps are discussed below.

After the downloading step is performed, the programmable hardware element is configured with a hardware implementation of the graphical program. At this point, in step 268 the image acquisition device (e.g., smart camera 190) may be optionally disconnected from the computer system 102, and may possibly be deployed for operation. For example, in the case of smart camera 190, the smart camera 190 may be disconnected from the computer system 102 and deployed in a vision application, such as a manufacturing vision inspection application, a manufacturing assembly application, or other vision application. If the image acquisition device is an image acquisition card or board that is designed to be located in a slot of the computer 102, the image acquisition card or board may optionally remain in the computer system 102 for use, or may be transferred to a different computer system. Alternatively, the method described above may be repetitively used to manufacture a plurality of image acquisition devices (e.g., smart camera 190) for later sale. As another alternative, the method described above may be used to configure an image acquisition device that has already been deployed, wherein the image acquisition device is configured over a network, i.e., where the hardware configuration is downloaded onto the image acquisition device (or other instrument) over a network.

After the programmable hardware element in the image acquisition device (e.g., smart camera 190) is configured with a hardware implementation of the graphical program, the image acquisition device can be used in an application. Thus the image acquisition device or smart camera 190 may acquire an image of an object in step 270, e.g., from a camera, or the camera in smart camera 190 may acquire the image. The programmable hardware element in the image acquisition device may then execute in step 272 to perform the image processing function on the image.

While the programmable hardware element in the image acquisition device executes to perform the image processing function on the image, if the graphical program includes a user interface portion, this user interface portion may optionally be presented on a display during this time in step 274. The code corresponding to the user interface portion may be executed by a processor in the computer system 102 or by a processor on the image acquisition device. The user interface portion may operate as a front panel for the image acquisition device. The user may optionally provide user input to the user interface portion on the display to control the image acquisition device while the programmable hardware element in the image acquisition device executes to perform the image processing function on the image.

FIG. 7--Block Diagram

FIG. 7 is a block diagram illustrating the conversion of a graphical program into hardware and software descriptions. The graphical program 300 comprises graphical code, such as interconnected function nodes or icons. The graphical code in the graphical program may use graphical data flow, graphical control flow and/or graphical execution flow constructs, as noted above. On the display, the graphical program is represented as interconnected icons or function nodes. In the memory of the computer system, the graphical program 300 may comprise data structures (or scripts or code) representing functional operations, data flow and/or control flow, and execution order. As the user assembles the graphical program on the display, e.g., by selecting, arranging, and connecting various icons or function nodes on the display, the data structures (or scripts or code) are automatically created and stored in memory.

The graphical program 300 may be created with various development tools. For example, the graphical program may be created using the following development systems: LabVIEW, BridgeVIEW, DASYLab, Visual Designer, HP VEE (Visual Engineering Environment), Snap-Master, GFS DiaDem, ObjectBench, Simulink, WiT, Vision Program Manager, Hypersignal, VisiDAQ, VisSim, Truly Visual, and Khoros, among others. In the preferred embodiment, graphical program 300 is a LabVIEW graphical program or virtual instrument (VI).

Programs of the present invention create the VDiagram tree 302 from the data structures of the graphical program 300. The VDiagram tree 302 is an abstract hardware graph which represents at least a portion of the graphical program 300. The graph is organized in a way that facilitates the generation of specific types of descriptions by back end programs of the present invention. In one embodiment, the graphical programming system automatically creates and stores a VDiagram tree 302 (abstract hardware graph) in response to a user's creation of a graphical program. In this instance, conversion from graphical program data structures to a VDiagram tree is not necessary.

A hardware description 304 may be generated from the abstract hardware graph 302 by a back end program. The hardware description 304 may be in various hardware description languages such as VHDL, EDIF, and Verilog. In the preferred embodiment, the hardware description 304 comprises one or more VHDL files. A hardware netlist 306 may be generated from the hardware description using various synthesis tools. As noted above, the term "netlist" comprises various intermediate hardware-specific description formats comprising information regarding the particular hardware elements required to implement a hardware design and the relationship among those elements. In the preferred embodiment, the hardware netlist 306 is an FPGA-specific netlist. The hardware netlist 306 is used to create or configure one or more functional hardware devices or hardware elements 308 which are configured to execute the portion of the graphical program 300 that is represented by the abstract hardware graph 302.

Hardware element 308 may comprise any of various devices. For example, hardware 308 may comprise a programmable logic device (PLD) such as an FPGA or CPLD. However, hardware 308 may comprise other types of hardware devices, such as a traditional circuit board which is created using the hardware netlist 306. In the preferred embodiment, hardware 308 is an interface card comprising an FPGA, wherein the interface card is comprised in the computer system where the graphical program 300 is created. The hardware 308 may also be comprised in an external device connected to the computer system where the graphical program 300 is created. The hardware 308 may be connected to the computer over an external serial or parallel bus, or over a network, such as the Internet.

As shown in FIG. 7, software description source code 310 may also be generated from the abstract hardware graph 302 by a back end program. The source code 310 may be in various source code languages such as C, C++, Java, etc. Machine code 312 may be produced from the source code 310 using various source code compilers. Linked machine code 314 may be produced from the machine code 312 using various machine code linkers. The linked machine code 314 is executable to perform the operations of the portion of the graphical program 300 that is represented by the abstract hardware graph 302.

FIG. 8--Block Diagram

FIG. 8 is a block diagram illustrating the generation of various types of hardware and software descriptions from a VDiagram tree. As described for FIG. 7, programs of the present invention create a VDiagram tree 302 from a graphical program 300. The VDiagram tree 302 represents at least a portion of the graphical program 300. Back end programs 330 generate hardware descriptions from the VDiagram tree 302. Exemplary back end programs 330A, 330B, and 330C are illustrated. Back end 330A generates a VHDL hardware description comprising one or more VHDL files. Back end 330B generates an EDIF hardware description comprising one or more EDIF files. Back end 330C generates a C source code software description comprising one or more C files.

The number and type of back end programs that may be present are not limited. In the preferred embodiment, one or more back end programs may be called automatically as part of a process initiated by a user to generate hardware/software descriptions for the graphical program 300. In another embodiment, the VDiagram tree 302 may be generated and saved to a file, and the user may call a back end program at a later time to generate a hardware/software description.

As described above for FIG. 7, appropriate synthesis tools or compilers may be called to convert a hardware/software description into another format such as an FPGA-specific netlist or compiled machine code.

FIG. 9--Block Diagram

FIG. 9 illustrates the exportation of at least a portion of a graphical program 300 into a hardware description and the use of the hardware description to program an FPGA. As described above for FIG. 7, the VDiagram tree 302 comprises information representing the graphical program 300, including the functional operations of the program. As described in detail below, the VDiagram tree comprises VDiagrams, each of which maintains a list of components. This list of components includes components which represent functional operations.

A back end program converts the VDiagram tree 302 to a hardware description 304. Back end programs may implement the functionality of the components in the VDiagram component lists using constructs of their respective description languages. For example, a VHDL back end may create VHDL code to implement a component that performs a particular mathematical algorithm such as an exponential calculation. However, in the preferred embodiment, such functional components are simply referenced as library components.

FIG. 9 illustrates the preferred embodiment in which the VDiagram tree references one or more library components. One embodiment of the present invention comprises pre-compiled function blocks 342 which implement these library components for particular hardware devices such as FPGAs. Various FPGA netlist synthesis tools may be called to generate an FPGA netlist 340 from the hardware description 304. These synthesis tools may incorporate the pre-compiled function blocks 342 into the FPGA netlist 340. Also, as shown, the synthesis tools may utilize hardware target-specific information in creating the netlist. For example, the exact form that the FPGA netlist takes may depend on the particular type of FPGA that will use the netlist, since FPGAs differ in their available resources.

An FPGA bit stream program file 346 may be generated from the FPGA netlist 340 using readily available synthesis tools. This FPGA program file may be uploaded to an FPGA 348. T