|
|
|
Graphical or iconic based (e.g., visual program) |
System and method for deploying a graphical program on an image acquisition device6971066
Abstract
A computer-implemented system and method for deploying a graphical program onto an image acquisition (IMAQ) device. The method may operate to configure an image acquisition (IMAQ) device to perform image processing or machine vision functions, wherein the device includes a programmable hardware element and/or a processor and memory. The method comprises first creating a graphical program which implements the image processing or machine vision function. A portion of the graphical program may be converted into a hardware implementation on a programmable hardware element, and a portion may optionally be compiled into machine code for execution by a CPU. The programmable hardware element is thus configured utilizing a hardware description and implements a hardware implementation of at least a portion of the graphical program. The CPU-executable code may be executed by a computer coupled to the IMAQ device, or by a processor/memory on the IMAQ device.
Claims
1. A computer-implemented method for configuring an image acquisition device to perform an image processing function, the method comprising:
creating a graphical program on a computer system, wherein the graphical program implements the image processing function, and wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
deploying the graphical program on the image acquisition device, wherein the image acquisition device includes a processor and a memory, wherein the image acquisition device is coupled to or comprised in the computer system;
wherein said deploying the graphical program on the image acquisition device comprises:
generating an executable program based on the graphical program, wherein the executable program implements functionality of the graphical program; and
transferring the executable program to the memory on the image acquisition device;
the method further comprising:
the image acquisition device acquiring an image of an object; and
the image acquisition device executing to perform the image processing function on the image, wherein said image acquisition device executing to perform the image processing function on the image comprises the processor in the image acquisition device executing the executable program from the memory.
2. The method of claim 1, further comprising:
the image acquisition device receiving an image present signal;
wherein the image acquisition device acquires the image of the object in response to receiving the image present signal.
3. The method of claim 1, further comprising:
the image acquisition device performing a control operation after said executing to perform the image processing function on the image.
4. The method of claim 1, further comprising:
the image acquisition device generating a pass/fail indication after said executing to perform the image processing function on the image.
5. The method of claim 1,
wherein the image acquisition device is coupled to a camera; and
wherein the image acquisition device acquiring an image of an object comprises the camera acquiring the image of the object and the image acquisition device receiving and storing the image.
6. The method of claim 1,
wherein said executing further comprises:
the image acquisition device determining characteristics of the image after performing the image processing function; and
the image acquisition device performing an operation based on the determined characteristics of the image.
7. The method of claim 1,
wherein the image acquisition device comprises a smart camera.
8. The method of claim 1,
wherein the image acquisition device comprises an image acquisition board coupled to or comprised in a computer system.
9. The method of claim 1,
wherein the graphical program includes a user interface portion;
the method further comprising:
presenting the user interface portion on a display during the image acquisition device executing to perform the image processing function on the image.
10. The method of claim 9,
wherein the user interface portion operates as a front panel for the image acquisition device.
11. The method of claim 9, further comprising:
receiving user input to the user interface portion on the display to control the image acquisition device during the processor in the image acquisition device executing to perform the image processing function on the image.
12. The method of claim 9, further comprising:
compiling the user interface portion into executable code for execution by a first processor and storing the executable code in a first memory; and
the first processor executing the executable code from the first memory to present the user interface portion on the display.
13. The method of claim 12,
wherein the image acquisition device includes the first processor and the first memory; and
wherein the first processor in the image acquisition device executes the executable code from the first memory to present the user interface portion on the display during the image acquisition device executing to perform the image processing function on the image.
14. The method of claim 12,
wherein the computer system includes the first processor and the first memory; and
wherein the computer system executes the executable code from the first memory to present the user interface portion on the display during the image acquisition device executing to perform the image processing function on the image.
15. The method of claim 1, wherein said creating the graphical program includes:
arranging on a display of the computer system a plurality of nodes comprising the graphical program and interconnecting the plurality of nodes;
creating and storing data structures which represent the graphical program in response to said arranging and said interconnecting.
16. The method of claim 1, wherein the graphical program comprises a data flow diagram.
17. The method of claim 1, wherein the graphical program comprises one or more of data flow, control flow and execution flow constructs.
18. The method of claim 1,
wherein said creating the graphical program is performed on the computer system;
wherein the image acquisition device is coupled to the computer system over a network; and
wherein said deploying the graphical program comprises the computer system deploying the graphical program over the network to the image acquisition device.
19. An image processing system, comprising:
a computer system comprising a processor, memory and a display;
wherein the memory stores a graphical program, wherein the graphical program implements an image processing function, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program; and
wherein the memory also stores a software program which is executable to deploy the graphical program on the image acquisition device; and
an image acquisition device coupled to the computer system, wherein the image acquisition device includes:
an input for acquiring an image; and
at least one functional unit that is configurable based on the graphical program for performing the image processing function on the image, wherein the at least one functional unit comprises a second processor and a second memory;
wherein the software program is executable by the processor in the computer system to transfer the graphical program to the second memory on the image acquisition device; and
wherein the second processor in the image acquisition device is operable to execute the graphical program from the second memory.
20. The image processing system of claim 19,
wherein the image acquisition device is operable to receive an image present signal; and
wherein the image acquisition device is operable to acquire the image of the object in response to receiving the image present signal.
21. The image processing system of claim 19,
wherein the image acquisition device is operable to perform a control operation after executing to perform the image processing function on the image.
22. The image processing system of claim 19,
wherein the image acquisition device is operable to generate a pass/fail indication after executing to perform the image processing function on the image.
23. The image processing system of claim 19,
wherein the image acquisition device is coupled to a camera; and
wherein the image acquisition device is operable to receive and store an image acquired from the camera.
24. The image processing system of claim 19,
wherein the image acquisition device is operable to determine characteristics of the image after performing the image processing function; and
wherein the image acquisition device is operable to perform an operation based on the determined characteristics of the image.
25. The image processing system of claim 19,
wherein the image acquisition device comprises a smart camera.
26. The image processing system of claim 19,
wherein the image acquisition device comprises an image acquisition board coupled to or comprised in a computer system.
27. The image processing system of claim 19,
wherein the graphical program includes a user interface portion; and
wherein the computer system is operable to present the user interface portion on a display when the at least one functional unit in the image acquisition device executes to perform the image processing function on the image.
28. The image processing system of claim 27,
wherein the user interface portion operates as a front panel for the image acquisition device.
29. The image processing system of claim 27,
wherein the computer system further comprises a user input device for receiving user input to the user interface portion on the display; and
wherein the user input is operable to control the image acquisition device when the at least one functional unit in the image acquisition device executes to perform the image processing function on the image.
30. The image processing system of claim 27,
wherein the user interface portion is operable to be compiled into executable code for execution by the processor and stored in the memory; and
wherein the processor is operable to execute the executable code from the memory to present the user interface portion on the display.
31. The image processing system of claim 19,
wherein the memory of the computer system stores a graphical program development program for creating the graphical program;
wherein the graphical program development program is executable to:
arrange on the screen a plurality of nodes comprising the graphical program and interconnect the plurality of nodes in response to user input; and
create and store data structures which represent the graphical program in response to said user input.
32. The image processing system of claim 19, wherein the graphical program comprises one or more of data flow, control flow and execution flow constructs.
33. The image processing system of claim 19,
wherein the image acquisition device is coupled to the computer system over a network; and
wherein the computer system is operable to deploy the graphical program over the network to the image acquisition device.
34. An image acquisition device, comprising:
an input for receiving an image of an object;
a memory coupled to the input for storing data corresponding to the image of the object;
a programmable hardware element coupled to the memory that is configured to implement an image processing function, wherein the programmable hardware element in the image acquisition device is configured utilizing a hardware description generated from a graphical program, wherein the programmable hardware element implements a hardware implementation of the graphical program, wherein the programmable hardware element in the image acquisition device is operable to perform an image processing function on the image; and
analog to digital conversion logic coupled to the input and to the programmable hardware element for performing analog to digital conversion logic on an acquired analog image to produce a digital image.
35. The image acquisition device of claim 34,
wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program.
36. The image acquisition device of claim 34,
wherein the graphical program is created by assembling a plurality of interconnected nodes which visually indicate functionality of the graphical program in response to user input.
37. A method for configuring an image acquisition device to perform an image processing function, the method comprising:
coupling the image acquisition device to a computer system, wherein the image acquisition device comprises at least one functional unit, wherein the functional unit is a processor and memory, wherein the computer system stores a graphical program, wherein the graphical program implements the image processing function; and
deploying the graphical program onto the functional unit in the image acquisition device to configure the functional unit, wherein said deploying the graphical program onto the functional unit comprises:
generating an executable program based on the graphical program, wherein the executable program implements functionality of the graphical program; and
transferring the executable program to the memory on the image acquisition device
wherein the processor in the image acquisition device is operable to execute the executable program from the memory; and
wherein after said deploying the functional unit is operable to implement the graphical program.
38. The method of claim 37, further comprising:
disconnecting the image acquisition device from the computer system after said deploying.
39. The method of claim 37, further comprising:
creating the graphical program prior to said deploying, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program.
40. The method of claim 37,
wherein said deploying the graphical program onto the functional unit further comprises transferring the graphical program to the memory of the functional unit; and
wherein the processor in the image acquisition device is operable to execute the graphical program from the memory.
41. A computer-implemented method for configuring a smart camera to perform an image processing function, the method comprising:
creating a graphical program on a computer system, wherein the graphical program implements the image processing function, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
deploying the graphical program on the smart camera, wherein the smart camera is coupled to the computer system;
the smart camera acquiring an image of an object; and
the smart camera executing to perform the image processing function on the image;
wherein the smart camera includes a processor and a memory;
wherein said deploying the graphical program on the smart camera comprises:
generating an executable program based on the graphical program, wherein the executable program implements functionality of the graphical program; and
transferring the executable program to the memory on the smart camera; and
wherein said smart camera executing to perform the image processing function on the image comprises the processor in the smart camera executing the executable program from the memory.
42. The method of claim 41,
wherein said deploying the graphical program on the smart camera comprises transferring the graphical program to the memory on the smart camera; and
wherein said smart camera executing to perform the image processing function on the image comprises the processor in the smart camera executing the graphical program from the memory.
43. The method of claim 41, further comprising:
the smart camera receiving an image present signal;
wherein the smart camera acquires the image of the object in response to receiving the image present signal.
44. The method of claim 41, further comprising:
the smart camera performing a control operation after said executing to perform the image processing function on the image.
45. The method of claim 41, further comprising:
the smart camera generating a pass/fail indication after said executing to perform the image processing function on the image.
46. The method of claim 41,
wherein the smart camera acquiring an image of an object comprises the smart camera acquiring the image of the object and storing the image in a memory.
47. The method of claim 41,
wherein said executing further comprises:
the smart camera determining characteristics of the image after performing the image processing function; and
the smart camera performing an operation based on the determined characteristics of the image.
48. The method of claim 41,
wherein the graphical program includes a user interface portion;
the method further comprising:
presenting the user interface portion on a display during the smart camera executing to perform the image processing function on the image.
49. The method of claim 48,
wherein the user interface portion operates as a front panel for the smart camera.
50. The method of claim 48, further comprising:
receiving user input to the user interface portion on the display to control the smart camera during the processor in the smart camera executing to perform the image processing function on the image.
51. The method of claim 48, further comprising:
compiling the user interface portion into executable code for execution by a processor and storing the executable code in a memory; and
the processor executing the executable code from the memory to present the user interface portion on the display.
52. The method of claim 51,
wherein the processor in the smart camera executes the executable code from the memory to present the user interface portion on the display during the smart camera executing to perform the image processing function on the image.
53. The method of claim 51,
wherein the computer system includes the processor and the memory; and
wherein the computer system executes the executable code from the memory to present the user interface portion on the display during the smart camera executing to perform the image processing function on the image.
54. The method of claim 41, wherein said creating the graphical program includes:
arranging on a display of the computer system a plurality of nodes comprising the graphical program and interconnecting the plurality of nodes; and
creating and storing data structures which represent the graphical program in response to said arranging and said interconnecting.
55. The method of claim 41, wherein the graphical program comprises a data flow diagram.
56. The method of claim 41, wherein the graphical program comprises one or more of data flow, control flow and execution flow constructs.
57. The method of claim 41,
wherein said creating the graphical program is performed on the computer system;
wherein the smart camera is coupled to the computer system over a network; and
wherein said deploying the graphical program comprises the computer system deploying the graphical program over the network to the smart camera.
58. An image processing system, comprising:
a computer system comprising a processor, memory and a display; and
a smart camera coupled to the computer system, wherein the smart camera includes:
a camera for acquiring an image; and
at least one functional unit coupled to the camera that is configurable based on the graphical program for performing the image processing function on the image, wherein the at least one functional unit comprises a second processor and a second memory;
wherein the memory of the computer system stores a graphical program, wherein the graphical program implements as image processing function, and wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
wherein the memory of the computer system also stores a software program which is executable to deploy the graphical program to the smart camera;
wherein the software program is executable by the processor in the computer system to transfer the graphical program to the second memory on the smart camera; and
wherein the second processor in the smart camera is operable to execute the graphical program from the second memory.
59. The image processing system of claim 58,
wherein the smart camera is operable to receive an image present signal; and
wherein the smart camera is operable to acquire the image of the object in response to receiving the image present signal.
60. The image processing system of claim 58,
wherein the smart camera is operable to perform a control operation after executing to perform the image processing function on the image.
61. The image processing system of claim 58,
wherein the smart camera includes a memory for storing an acquired image.
62. The image processing system of claim 58,
wherein the smart camera is operable to determine characteristics of the image after performing the image processing function; and
wherein the smart camera is operable to perform an operation based on the determined characteristics of the image.
63. The image processing system of claim 58,
wherein the graphical program includes a user interface portion; and
wherein the computer system is operable to present the user interface portion on a display when the at least one functional unit in the smart camera executes to perform the image processing function on the image.
64. The image processing system of claim 63,
wherein the user interface portion operates as a front panel for the smart camera.
65. The image processing system of claim 63,
wherein the computer system further comprises a user input device for receiving user input to the user interface portion on the display; and
wherein the user input is operable to control the smart camera when the at least one functional unit in the smart camera executes to perform the image processing function on the image.
66. The image processing system of claim 63,
wherein the user interface portion is operable to be compiled into executable code for execution by the processor and stored in the memory; and
wherein the processor is operable to execute the executable code from the memory to present the user interface portion on the display.
67. The image processing system of claim 58,
wherein the memory of the computer system stores a graphical program development program for creating the graphical program; and
wherein the graphical program development program is executable to:
arrange on the screen a plurality of nodes comprising the graphical program and interconnect the plurality of nodes in response to user input; and
create and store data structures which represent the graphical program in response to said user input.
68. The image processing system of claim 58, wherein the graphical program comprises one or more of data flow, control flow and execution flow constructs.
69. The image processing system of claim 58,
wherein the smart camera is coupled to the computer system over a network; and
wherein the computer system is operable to deploy the graphical program over the network to the smart camera.
70. A smart camera, comprising:
a camera for receiving an image of an object;
a memory coupled to the camera for storing data corresponding to the image of the object;
a programmable hardware element coupled to the memory that is configured to implement an image processing function, wherein the programmable hardware element in the smart camera is configured utilizing a hardware description generated from a graphical program, wherein the programmable hardware element implements a hardware implementation of the graphical program, and wherein the programmable hardware element in the smart camera is operable to perform an image processing function on the image; and
analog to digital conversion logic coupled to the input and to the programmable hardware element for performing analog to digital conversion logic on an acquired analog image to produce a digital image.
71. The smart camera of claim 70,
wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program.
72. The smart camera of claim 70,
wherein the graphical program is created by assembling a plurality of interconnected nodes which visually indicate functionality of the graphical program in response to user input.
73. A method for configuring a smart camera to perform an image processing function, the method comprising:
coupling the smart camera to a computer system, wherein the smart camera comprises at least one functional unit, wherein the functional unit is a processor and memory, wherein the computer system stores a graphical program, wherein the graphical program implements the image processing function; and
deploying the graphical program onto the functional unit in the smart camera to configure the functional unit, wherein said deploying the graphical program onto the functional unit comprises:
generating an executable program based on the graphical program, wherein the executable program implements functionality of the graphical program; and
transferring the executable program to the memory on the smart camera;
wherein after said deploying the functional unit is operable to implement the graphical program; and
wherein the processor in the smart camera is operable to execute the executable program from the memory.
74. The method of claim 73, further comprising:
disconnecting the smart camera from the computer system after said deploying.
75. The method of claim 73, further comprising:
creating the graphical program prior to said deploying, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program.
76. The method of claim 73,
wherein said deploying the graphical program onto the functional unit comprises transferring the graphical program to the memory of the functional unit; and
wherein the processor in the smart camera is operable to execute the graphical program from the memory.
77. A computer-implemented method for configuring an image acquisition device to perform an image processing function, the method comprising:
creating a graphical program on a computer system, wherein the graphical program implements the image processing function, and wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
deploying the graphical program on the image acquisition device, wherein the image acquisition device includes a programmable hardware element, wherein the image acquisition device is coupled to or comprised in the computer system;
the image acquisition device acquiring an image of an object; and
the image acquisition device executing to perform the image processing function on the image;
wherein said deploying the graphical program on the image acquisition device comprises:
generating a hardware description based on the graphical program, wherein the hardware description describes a hardware implementation of the graphical program; and
configuring the programmable hardware element in the image acquisition device utilizing the hardware description, wherein after said configuring the programmable hardware element implements a hardware implementation of the graphical program; and
wherein said image acquisition device executing to perform the image processing function on the image comprises the programmable hardware element in the image acquisition device executing to perform the image processing function on the image.
78. A computer-implemented method for configuring an image acquisition device to perform an image processing function, the method comprising:
creating a graphical program on a computer system, wherein the graphical program implements the image processing function, and wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
deploying the graphical program on the image acquisition device, wherein the image acquisition device includes a processor and a memory and a programmable hardware element, wherein the image acquisition device is coupled to or comprised in the computer system;
the image acquisition device acquiring an image of an object; and
the image acquisition device executing to perform the image processing function on the image;
wherein said deploying the graphical program on the image acquisition device comprises:
transferring a first portion of the graphical program to the memory on the image acquisition device;
generating a hardware description based on a second portion of the graphical program, wherein the hardware description describes a hardware implementation of the second portion of the graphical program; and
configuring the programmable hardware element in the image acquisition device utilizing the hardware description, wherein after said configuring the programmable hardware element implements a hardware implementation of the second portion of the graphical program; and
wherein said image acquisition device executing to perform the image processing function on the image comprises the processor in the image acquisition device executing the first portion of the graphical program from the memory and the programmable hardware element executing the second portion of the graphical program.
79. The method of claim 78,
wherein the first portion of the graphical program comprises an image processing portion, and wherein the second portion of the graphical program comprises a control portion.
80. The method of claim 79,
wherein the first portion of the graphical program comprises a control portion, and wherein the second portion of the graphical program comprises an image processing portion.
81. A computer-implemented method for configuring an image acquisition device to perform an image processing function, the method comprising:
creating a graphical program on a computer system, wherein the graphical program implements the image processing function, and wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
deploying the graphical program on the image acquisition device, wherein the image acquisition device includes a processor and a memory and a programmable hardware element, wherein the image acquisition device is coupled to or comprised in the computer system;
the image acquisition device acquiring an image of an object; and
the image acquisition device executing to perform the image processing function on the image;
wherein said deploying the graphical program on the image acquisition device comprises:
generating an executable program based on a first portion of the graphical program, wherein the executable program implements functionality of the first portion of the graphical program; and
transferring the executable program to the memory on the image acquisition device;
generating a hardware description based on a second portion of the graphical program, wherein the hardware description describes a hardware implementation of the graphical program; and
configuring the programmable hardware element in the image acquisition device utilizing the hardware description, wherein after said configuring the programmable hardware element implements a hardware implementation of the second portion of the graphical program; and
wherein said image acquisition device executing to perform the image processing function on the image comprises the processor in the image acquisition device executing the executable program from the memory and the programmable hardware element executing the second portion of the graphical program.
82. The method of claim 81,
wherein the first portion of the graphical program comprises an image processing portion, and wherein the second portion of the graphical program comprises a control portion.
83. The method of claim 81,
wherein the first portion of the graphical program comprises a control portion, and wherein the second portion of the graphical program comprises an image processing portion.
84. A computer-implemented method for configuring an image acquisition device to perform an image processing function, the method comprising:
creating a graphical program on a computer system, wherein the graphical program implements the image processing function, and wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
deploying the graphical program on the image acquisition device, wherein the image acquisition device includes a first processor, a second processor, and at least one memory, wherein the image acquisition device is coupled to or comprised in the computer system;
the image acquisition device acquiring an image of an object; and
the image acquisition device executing to perform the image processing function on the image.
wherein said image acquisition device executing to perform the image processing function on the image comprises the first processor in the image acquisition device executing a first portion of the graphical program from the at least one memory and the second processor in the image acquisition device executing a second portion of the graphical program from the at least one memory.
85. The method of claim 84,
wherein the first portion of the graphical program comprises an image processing portion, and wherein the second portion of the graphical program comprises a control portion.
86. The method of claim 84,
wherein the first portion of the graphical program comprises a first portion of the image processing function, and wherein the second portion of the graphical program comprises a second portion of the image processing function.
87. A computer-implemented method for configuring an image acquisition device to perform an image processing function, the method comprising:
creating a graphical program on a computer system, wherein the graphical program implements the image processing function, and wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
deploying the graphical program on the image acquisition device, wherein the image acquisition device is coupled to or comprised in the computer system;
the image acquisition device acquiring an image of an object;
the image acquisition device executing to perform the image processing function on the image; and
the image acquisition device generating a pass/fail indication after said executing to perform the image processing function on the image.
88. A computer-implemented method for configuring an image acquisition device to perform an image processing function, the method comprising:
creating a graphical program on a computer system, wherein the graphical program implements the image processing function, wherein the graphical program includes a user interface portion, and wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
deploying the graphical program on the image acquisition device, wherein the image acquisition device is coupled to or comprised in the computer system;
the image acquisition device acquiring an image of an object;
the image acquisition device executing to perform the image processing function on the image; and
presenting the user interface portion on a display during the image acquisition device executing to perform the image processing function on the image;
wherein the user interface portion operates as a front panel for the image acquisition device.
89. A computer-implemented method for configuring an image acquisition device to perform an image processing function, the method comprising:
creating a graphical program on a computer system, wherein the graphical program implements the image processing function, wherein the graphical program includes a user interface portion, and wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
deploying the graphical program on the image acquisition device, wherein the image acquisition device is coupled to or comprised in the computer system;
the image acquisition device acquiring an image of an object;
the image acquisition device executing to perform the image processing function on the image;
presenting the user interface portion on a display during the image acquisition device executing to perform the image processing function on the image;
compiling the user interface portion into executable code for execution by a processor and storing the executable code in a memory; and
the processor executing the executable code from the memory to present the user interface portion on the display.
90. The method of claim 89,
wherein the image acquisition device includes the processor and the memory; and
wherein the processor in the image acquisition device executes the executable code from the memory to present the user interface portion on the display during the image acquisition device executing to perform the image processing function on the image.
91. The method of claim 89,
wherein the computer system includes the processor and the memory; and
wherein the computer system executes the executable code from the memory to present the user interface portion on the display during the image acquisition device executing to perform the image processing function on the image.
92. An image processing system, comprising:
a computer system comprising a processor, memory and a display; and
an image acquisition device coupled to the computer system, wherein the image acquisition device includes:
an input for acquiring an image; and
at least one functional unit that is configurable based on the graphical program for performing the image processing function on the image, wherein the at least one functional unit comprises a second processor and a second memory;
wherein the memory of the computer system stores a graphical program, wherein the graphical program implements an image processing function, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
wherein the memory of the computer system also stores a software program which is executable to deploy the graphical program on the image acquisition device;
wherein the software program is executable by the processor in the computer system to 1) generate an executable program based on the graphical program, wherein the executable program implements functionality of the graphical program, and 2) transfer the executable program to the second memory on the image acquisition device; and
wherein the second processor in the image acquisition device is operable to execute the executable program from the second memory.
93. An image processing system, comprising:
a computer system comprising a processor, memory and a display; and
an image acquisition device coupled to the computer system, wherein the image acquisition device includes:
an input for acquiring an image; and
at least one functional unit that is configurable based on the graphical program for performing the image processing function on the image, wherein the at least one functional unit comprises a programmable hardware element;
wherein the memory of the computer system stores a graphical program, wherein the graphical program implements an image processing function, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
wherein the memory of the computer system also stores a software program which is executable to deploy the graphical program on the image acquisition device; and
wherein the software program is executable by the processor in the computer system to generate a hardware description based on the graphical program and configure the programmable hardware element based on the hardware description, wherein after being configured the programmable hardware element, implements a hardware implementation of the graphical program, wherein the programmable hardware element in the image acquisition device is executable to perform an image processing function on an acquired image.
94. An image processing system, comprising:
a computer system comprising a processor, memory and a display; and
an image acquisition device coupled to the computer system, wherein the image acquisition device includes:
an input for acquiring an image; and
at least one functional unit that is configurable based on the graphical program for performing the image processing function on the image, wherein the at least one functional unit comprises a second processor, a second memory, and a programmable hardware element;
wherein the memory of the computer system stores a graphical program, wherein the graphical program implements an image processing function, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
wherein the memory of the computer system also stores a software program which is executable to deploy the graphical program on the image acquisition device;
wherein the software program is executable by the processor in the computer system to:
transfer a first portion of the graphical program to the second memory on the image acquisition device;
generate a hardware description based on a second portion of the graphical program, wherein the hardware description describes a hardware implementation of the second portion of the graphical program; and
configure the programmable hardware element in the image acquisition device utilizing the hardware description, wherein after said configuring the programmable hardware element implements a hardware implementation of the second portion of the graphical program;
wherein the second processor in the image acquisition device is operable to execute the first portion of the graphical program from the second memory; and
wherein the programmable hardware element in the image acquisition device is operable to implement the second portion of the graphical program.
95. The image processing system of claim 94,
wherein the first portion of the graphical program comprises an image processing portion, and wherein the second portion of the graphical program comprises a control portion.
96. The image processing system of claim 94,
wherein the first portion of the graphical program comprises a control portion, and wherein the second portion of the graphical program comprises an image processing portion.
97. An image processing system, comprising:
a computer system comprising a processor, memory and a display; and
an image acquisition device coupled to the computer system, wherein the image acquisition device includes:
an input for acquiring an image; and
at least one functional unit that is configurable based on the graphical program for performing the image processing function on the image, wherein the at least one functional unit comprises a second processor, a second memory, and a programmable hardware element;
wherein the memory of the computer system stores a graphical program, wherein the graphical program implements an image processing function, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
wherein the memory of the computer system also stores a software program which is executable to deploy the graphical program on the image acquisition device;
wherein the software program is executable by the processor in the computer system to:
generate an executable program based on a first portion of the graphical program, wherein the executable program implements functionality of the first portion of the graphical program;
transfer the executable program to the second memory on the image acquisition device;
generate a hardware description based on a second portion of the graphical program, wherein the hardware description describes a hardware implementation of the second portion of the graphical program; and
configure the programmable hardware element in the image acquisition device utilizing the hardware description, wherein after said configuring the programmable hardware element implements a hardware implementation of the second portion of the graphical program;
wherein the second processor in the image acquisition device is operable to execute the executable program from the second memory to implement the first portion of the graphical program; and
wherein the programmable hardware element in the image acquisition device is operable to implement the second portion of the graphical program.
98. The image processing system of claim 97,
wherein the first portion of the graphical program comprises an image processing portion, and wherein the second portion of the graphical program comprises a control portion.
99. The image processing system of claim 96,
wherein the first portion of the graphical program comprises a control portion, and wherein the second portion of the graphical program comprises an image processing portion.
100. An image processing system, comprising:
a computer system comprising a processor, memory and a display; and
an image acquisition device coupled to the computer system, wherein the image acquisition device includes:
an input for acquiring an image; and
at least one functional unit that is configurable based on the graphical program for performing the image processing function on the image, wherein the at least one functional unit comprises a second processor, a third processor, and at least one memory;
wherein the memory of the computer system stores a graphical program, wherein the graphical program implements an image processing function, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
wherein the memory of the computer system also stores a software program which is executable to deploy the graphical program on the image acquisition device; and
wherein the second processor in the image acquisition device is operable to execute a first portion of the graphical program from the at least one memory and the third processor in the image acquisition device is operable to execute a second portion of the graphical program from the at least one memory.
101. The image processing system of claim 100,
wherein the first portion of the graphical program comprises an image processing portion, and wherein the second portion of the graphical program comprises a control portion.
102. The image processing system of claim 100,
wherein the first portion of the graphical program comprises a first portion of the image processing function, and wherein the second portion of the graphical program comprises a second portion of the image processing function.
103. An image processing system, comprising:
a computer system comprising a processor, memory and a display; and
an image acquisition device coupled to the computer system, wherein the image acquisition device includes:
an input for acquiring an image; and
at least one functional unit that is configurable based on the graphical program for performing the image processing function on the image;
wherein the memory of the computer system stores a graphical program, wherein the graphical program implements an image processing function, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
wherein the memory of the computer system also stores a software program which is executable to deploy the graphical program on the image acquisition device; and
wherein the image acquisition device is operable to generate a pass/fail indication after executing to perform the image processing function on the image.
104. An image processing system, comprising:
a computer system comprising a processor, memory and a display; and
an image acquisition device coupled to the computer system, wherein the image acquisition device includes:
an input for acquiring an image; and
at least one functional unit that is configurable based on the graphical program for performing the image processing function on the image;
wherein the memory of the computer system stores a graphical program, wherein the graphical program implements an image processing function, wherein the graphical program includes a user interface portion, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
wherein the memory of the computer system also stores a software program which is executable to deploy the graphical program on the image acquisition device;
wherein the computer system is operable to present the user interface portion on a display when the at least one functional unit in the image acquisition device executes to perform the image processing function on the image; and
wherein the user interface portion operates as a front panel for the image acquisition device.
105. An image processing system, comprising:
a computer system comprising a processor, memory and a display; and
an image acquisition device coupled to the computer system, wherein the image acquisition device includes:
an input for acquiring an image; and
at least one functional unit that is configurable based on the graphical program for performing the image processing function on the image;
wherein the memory of the computer system stores a graphical program, wherein the graphical program implements an image processing function, wherein the graphical program includes a user interface portion, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program; and
wherein the memory of the computer system also stores a software program which is executable to deploy the graphical program on the image acquisition device;
wherein the user interface portion is operable to be compiled into executable code for execution by the processor and stored in the memory;
wherein the processor is operable to execute the executable code from the memory to present the user interface portion on the display; and
wherein the computer system is operable to present the user interface portion on a display when the at least one functional unit in the image acquisition device executes to perform the image processing function on the image.
106. An image acquisition device, comprising:
an input for receiving an image of an object;
a memory coupled to the input for storing data corresponding to the image of the object;
a programmable hardware element coupled to the memory that is configured to implement an image processing function, wherein the programmable hardware element in the image acquisition device is configured utilizing a hardware description generated from a graphical program, wherein the programmable hardware element implements a hardware implementation of the graphical program, wherein the programmable hardware element in the image acquisition device is operable to perform an image processing function on the image; and
timer/counter logic, wherein the timer/counter logic performs one of timing/counting operations while the programmable hardware element in the image acquisition device executes to perform the image processing function on the image.
107. The image acquisition device of claim 106,
wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program.
108. An image acquisition device, comprising:
an input for receiving an image of an object;
a memory coupled to the input for storing data corresponding to the image of the object; and
a programmable hardware element coupled to the memory that is configured to implement an image processing function, wherein the programmable hardware element in the image acquisition device is configured utilizing a hardware description generated from a graphical program, wherein the programmable hardware element implements a hardware implementation of the graphical program, wherein the programmable hardware element in the image acquisition device is operable to perform an image processing function on the image, wherein the programmable hardware element comprises a field programmable gate array (FPGA).
109. The image acquisition device of claim 108,
wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program.
110. An image acquisition device, comprising:
an input for receiving an image of an object;
a memory coupled to the input for storing data corresponding to the image of the object;
a programmable hardware element coupled to the memory that is configured to implement an image processing function, wherein the programmable hardware element in the image acquisition device is configured utilizing a hardware description generated from a graphical program, wherein the programmable hardware element implements a hardware implementation of the graphical program, wherein the programmable hardware element in the image acquisition device is operable to perform an image processing function on the image; and
a non-volatile memory coupled to the programmable hardware element, wherein the non-volatile memory is operable to store the hardware description, wherein the non-volatile memory is further operable to transfer the hardware description to the programmable hardware element to configure the programmable hardware element.
111. The image acquisition device of claim 110,
wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program.
112. A method for configuring an image acquisition device to perform an image processing function, the method comprising:
coupling the image acquisition device to a computer system, wherein the image acquisition device comprises at least one functional unit, wherein the functional unit is a programmable hardware element, wherein the computer system stores a graphical program, wherein the graphical program implements the image processing function; and
deploying the graphical program onto the functional unit in the image acquisition device to configure the functional unit, wherein said deploying the graphical program onto the functional unit comprises:
downloading a hardware configuration onto the programmable hardware element in the image acquisition device to configure the programmable hardware element, wherein the hardware configuration corresponds to a hardware implementation of the graphical program, wherein after said downloading the programmable hardware element implements a hardware implementation of the graphical program;
wherein after said deploying the functional unit is operable to implement the graphical program.
113. The method of claim 112, further comprising:
creating the graphical program prior to said deploying, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program.
114. The method of claim 112, further comprising:
disconnecting the image acquisition device from the computer system after said deploying.
115. A computer-implemented method for configuring a smart camera to perform an image processing function, the method comprising:
creating a graphical program on a computer system, wherein the graphical program implements the image processing function, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
deploying the graphical program on the smart camera, wherein the smart camera is coupled to the computer system;
the smart camera acquiring an image of an object; and
the smart camera executing to perform the image processing function on the image;
wherein the smart camera includes a programmable hardware element;
wherein said deploying the graphical program on the smart camera comprises:
generating a hardware description based on the graphical program, wherein the hardware description describes a hardware implementation of the graphical program; and
configuring the programmable hardware element in the smart camera utilizing the hardware description, wherein after said configuring the programmable hardware element implements a hardware implementation of the graphical program; and
wherein said smart camera executing to perform the image processing function on the image comprises the programmable hardware element in the smart camera executing to perform the image processing function on the image.
116. A computer-implemented method for configuring a smart camera to perform an image processing function, the method comprising:
creating a graphical program on a computer system, wherein the graphical program implements the image processing function, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
deploying the graphical program on the smart camera, wherein the smart camera is coupled to the computer system;
the smart camera acquiring an image of an object; and
the smart camera executing to perform the image processing function on the image;
wherein the smart camera includes a processor and a memory and a programmable hardware element;
wherein said deploying the graphical program on the smart camera comprises:
transferring a first portion of the graphical program to the memory on the smart camera;
generating a hardware description based on a second portion of the graphical program, wherein the hardware description describes a hardware implementation of the second portion of the graphical program; and
configuring the programmable hardware element in the smart camera utilizing the hardware description, wherein after said configuring the programmable hardware element implements a hardware implementation of the second portion of the graphical program; and
wherein said smart camera executing to perform the image processing function on the image comprises the processor in the smart camera executing the first portion of the graphical program from the memory and the programmable hardware element executing the second portion of the graphical program.
117. The method of claim 116,
wherein the first portion of the graphical program comprises an image processing portion, and wherein the second portion of the graphical program comprises a control portion.
118. The method of claim 116,
wherein the first portion of the graphical program comprises a control portion, and wherein the second portion of the graphical program comprises an image processing portion.
119. A computer-implemented method for configuring a smart camera to perform an image processing function, the method comprising:
creating a graphical program on a computer system, wherein the graphical program implements the image processing function, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
deploying the graphical program on the smart camera, wherein the smart camera is coupled to the computer system;
the smart camera acquiring an image of an object; and
the smart camera executing to perform the image processing function on the image;
wherein the smart camera includes a processor and a memory and a programmable hardware element;
wherein said deploying the graphical program on the smart camera comprises:
generating an executable program based on a first portion of the graphical program, wherein the executable program implements functionality of the first portion of the graphical program;
transferring the executable program to the memory on the smart camera;
generating a hardware description based on a second portion of the graphical program, wherein the hardware description describes a hardware implementation of the second portion of the graphical program; and
configuring the programmable hardware element in the smart camera utilizing the hardware description, wherein after said configuring the programmable hardware element implements a hardware implementation of the second portion of the graphical program; and
wherein the smart camera executing to perform the image processing function on the image comprises the processor in the smart camera executing the executable program from the memory and the programmable hardware element executing the second portion of the graphical program.
120. The method of claim 119,
wherein the first portion of the graphical program comprises an image processing portion, and wherein the second portion of the graphical program comprises a control portion.
121. The method of claim 119,
wherein the first portion of the graphical program comprises a control portion, and wherein the second portion of the graphical program comprises an image processing portion.
122. A computer-implemented method for configuring a smart camera to perform an image processing function, the method comprising:
creating a graphical program on a computer system, wherein the graphical program implements the image processing function, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
deploying the graphical program on the smart camera, wherein the smart camera is coupled to the computer system;
the smart camera acquiring an image of an object; and
the smart camera executing to perform the image processing function on the image;
wherein the smart camera includes a first processor, a second processor, and at least one memory; and
wherein said smart camera executing to perform the image processing function on the image comprises the first processor in the smart camera executing a first portion of the graphical program from the at least one memory and the second processor in the smart camera executing a second portion of the graphical program from the at least one memory.
123. The method of claim 122,
wherein the first portion of the graphical program comprises an image processing portion, and wherein the second portion of the graphical program comprises a control portion.
124. The method of claim 122,
wherein the first portion of the graphical program comprises a first portion of the image processing function, and wherein the second portion of the graphical program comprises a second portion of the image processing function.
125. A computer-implemented method for configuring a smart camera to perform an image processing function, the method comprising:
creating a graphical program on a computer system, wherein the graphical program implements the image processing function, wherein the graphical program includes a user interface portion, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
deploying the graphical program on the smart camera, wherein the smart camera is coupled to the computer system;
the smart camera acquiring an image of an object;
the smart camera executing to perform the image processing function on the image; and
presenting the user interface portion on a display during the smart camera executing to perform the image processing function on the image;
wherein the user interface portion operates as a front panel for the smart camera.
126. A computer-implemented method for configuring a smart camera to perform an image processing function, the method comprising:
creating a graphical program on a computer system, wherein the graphical program implements the image processing function, wherein the graphical program includes a user interface portion, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
compiling the user interface portion into executable code for execution by a processor and storing the executable code in a memory
deploying the graphical program on the smart camera, wherein the smart camera is coupled to the computer system;
the smart camera acquiring an image of an object;
the smart camera executing to perform the image processing function on the image;
the processor executing the executable code from the memory to present the user interface portion on the display; and
presenting the user interface portion on a display during the smart camera executing to perform the image processing function on the image.
127. The method of claim 126,
wherein the smart camera includes the processor and the memory; and
wherein the processor in the smart camera executes the executable code from the memory to present the user interface portion on the display during the smart camera executing to perform the image processing function on the image.
128. The method of claim 126,
wherein the computer system includes the processor and the memory; and
wherein the computer system executes the executable code from the memory to present the user interface portion on the display during the smart camera executing to perform the image processing function on the image.
129. An image processing system, comprising:
a computer system comprising a processor, memory and a display; and
a smart camera coupled to the computer system, wherein the smart camera includes:
a camera for acquiring an image; and
at least one functional unit coupled to the camera that is configurable based on the graphical program for performing the image processing function on the image, wherein the at least one functional unit comprises a second processor and a second memory;
wherein the memory of the computer system stores a graphical program, wherein the graphical program implements an image processing function, and wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program; and
wherein the memory of the computer system also stores a software program which is executable to deploy the graphical program to the smart camera;
wherein the software program is executable by the processor in the computer system to 1) generate an executable program based on the graphical program, wherein the executable program implements functionality of the graphical program, and 2) transfer the graphical program to the second memory of the at least one functional unit; and
wherein the second processor of the at least one functional unit is operable to execute the executable program from the second memory of the at least one functional unit.
130. An image processing system, comprising:
a computer system comprising a processor, memory and a display; and
a smart camera coupled to the computer system, wherein the smart camera includes:
a camera for acquiring an image; and
at least one functional unit coupled to the camera that is configurable based on the graphical program for performing the image processing function on the image, wherein the at least one functional unit comprises a programmable hardware element;
wherein the memory of the computer system stores a graphical program, wherein the graphical program implements an image processing function, and wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program; and
wherein the memory of the computer system also stores a software program which is executable to deploy the graphical program to the smart camera; and
wherein the software program is executable by the processor in the computer system to generate a hardware description based on the graphical program and configure the programmable hardware element based on the hardware description, wherein after being configured the programmable hardware element implements a hardware implementation of the graphical program, and wherein the programmable hardware element in the smart camera is executable to perform an image processing function on an acquired image.
131. An image processing system, comprising:
a computer system comprising a processor, memory and a display; and
a smart camera coupled to the computer system, wherein the smart camera includes:
a camera for acquiring an image; and
at least one functional unit coupled to the camera that is configurable based on the graphical program for performing the image processing function on the image, wherein the at least one functional unit comprises a second processor, a second memory, and a programmable hardware element;
wherein the memory of the computer system stores a graphical program, wherein the graphical program implements an image processing function, and wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
wherein the memory of the computer system also stores a software program which is executable to deploy the graphical program to the smart camera; wherein the software program is executable by the processor in the computer system to:
transfer a first portion of the graphical program to the second memory the smart camera;
generate a hardware description based on a second portion of the graphical program, wherein the hardware description describes a hardware implementation of the second portion of the graphical program; and
configure the programmable hardware element in the smart camera utilizing the hardware description, wherein after said configuring the programmable hardware element implements a hardware implementation of the second portion of the graphical program;
wherein the second processor in the smart camera is operable to execute the first portion of the graphical program from the second memory; and
wherein the programmable hardware element in the smart camera is operable to implement the second portion of the graphical program.
132. The image processing system of claim 131,
wherein the first portion of the graphical program comprises an image processing portion, and wherein the second portion of the graphical program comprises a control portion.
133. The image processing system of claim 131,
wherein the first portion of the graphical program comprises a control portion, and wherein the second portion of the graphical program comprises an image processing portion.
134. An image processing system, comprising:
a computer system comprising a processor, memory and a display; and
a smart camera coupled to the computer system, wherein the smart camera includes:
a camera for acquiring an image; and
at least one functional unit coupled to the camera that is configurable based on the graphical program for performing the image processing function on the image, wherein the at least one functional unit comprises a second processor, a second memory, and a programmable hardware element;
wherein the memory of the computer system stores a graphical program, wherein the graphical program implements an image processing function, and wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
wherein the memory of the computer system also stores a software program which is executable to deploy the graphical program to the smart camera;
wherein the software program is executable by the processor in the computer system to:
generate an executable program based on a first portion of the graphical program, wherein the executable program implements functionality of the first portion of the graphical program;
transfer the executable program to the second memory on the smart camera;
generate a hardware description based on a second portion of the graphical program, wherein the hardware description describes a hardware implementation of the second portion of the graphical program; and
configure the programmable hardware element in the smart camera utilizing the hardware description, wherein after said configuring the programmable hardware element implements a hardware implementation of the second portion of the graphical program;
wherein the second processor in the smart camera is operable to execute the executable program from the second memory; and
wherein the programmable hardware element in the smart camera is operable to implement the second portion of the graphical program.
135. The image processing system of claim 134,
wherein the first portion of the graphical program comprises an image processing portion, and wherein the second portion of the graphical program comprises a control portion.
136. The image processing system of claim 134,
wherein the first portion of the graphical program comprises a control portion, and wherein the second portion of the graphical program comprises an image processing portion.
137. An image processing system, comprising:
a computer system comprising a processor, memory and a display; and
a smart camera coupled to the computer system, wherein the smart camera includes:
a camera for acquiring an image; and
at least one functional unit coupled to the camera that is configurable based on the graphical program for performing the image processing function on the image, wherein the at least one functional unit comprises a second processor, a third processor, and at least one memory;
wherein the memory of the computer system stores a graphical program, wherein the graphical program implements an image processing function, and wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
wherein the memory of the computer system also stores a software program which is executable to deploy the graphical program on the smart camera; and
wherein the second processor of the at least one functional unit is operable to execute a first portion of the graphical program from the at least one memory of the at least one functional unit and the third processor of the at least one functional unit is operable to execute a second portion of the graphical program from the at least one memory of the at least one functional unit.
138. The image processing system of claim 137,
wherein the first portion of the graphical program comprises an image processing portion, and wherein the second portion of the graphical program comprises a control portion.
139. The image processing system of claim 137,
wherein the first portion of the graphical program comprises a first portion of the image processing function, and wherein the second portion of the graphical program comprises a second portion of the image processing function.
140. An image processing system, comprising:
a computer system comprising a processor, memory and a display; and
a smart camera coupled to the computer system, wherein the smart camera includes:
a camera for acquiring an image; and
at least one functional unit coupled to the camera that is configurable based on the graphical program for performing the image processing function on the image;
wherein the memory of the computer system stores a graphical program, wherein the graphical program implements an image processing function, and wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
wherein the memory of the computer system also stores a software program which is executable to deploy the graphical program to the smart camera; and
wherein the smart camera is operable to generate a pass/fail indication after executing to perform the image processing function on the image.
141. An image processing system, comprising:
a computer system comprising a processor, memory and a display; and
a smart camera coupled to the computer system, wherein the smart camera includes:
a camera for acquiring an image; and
at least one functional unit coupled to the camera that is configurable based on the graphical program for performing the image processing function on the image;
wherein the memory of the computer system stores a graphical program, wherein the graphical program implements an image processing function, wherein the graphical program includes a user interface portion, and wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
wherein the memory of the computer system also stores a software program which is executable to deploy the graphical program to the smart camera;
wherein the computer system is operable to present the user interface portion on a display when the at least one functional unit in the smart camera executes to perform the image processing function on the image; and
wherein the user interface portion operates as a front panel for the smart camera.
142. An image processing system, comprising:
a computer system comprising a processor, memory and a display; and
a smart camera coupled to the computer system, wherein the smart camera includes:
a camera for acquiring an image; and
at least one functional unit coupled to the camera that is configurable based on the graphical program for performing the image processing function on the image;
wherein the memory of the computer system stores a graphical program, wherein the graphical program implements an image processing function, wherein the graphical program includes a user interface portion, and wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program;
wherein the memory of the computer system also stores a software program which is executable to deploy the graphical program to the smart camera;
wherein the computer system is operable to present the user interface portion on a display when the at least one functional unit in the smart camera executes to perform the image processing function on the image;
wherein the user interface portion is operable to be compiled into executable code for execution by the processor of the computer system and stored in the memory of the computer system; and
wherein the processor of the computer system is operable to execute the executable code from the memory of the computer system to present the user interface portion on the display.
143. A smart camera, comprising:
a camera for receiving an image of an object;
a memory coupled to the camera for storing data corresponding to the image of the object;
a programmable hardware element coupled to the memory that is configured to implement an image processing function, wherein the programmable hardware element in the smart camera is configured utilizing a hardware description generated from a graphical program, wherein the programmable hardware element implements a hardware implementation of the graphical program, and wherein the programmable hardware element in the smart camera is operable to perform an image processing function on the image; and
timer/counter logic;
wherein the timer/counter logic performs one of timing/counting operations while the programmable hardware element in the smart camera executes to perform the image processing function on the image.
144. The smart camera of claim 143,
wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program.
145. A smart camera, comprising:
a camera for receiving an image of an object;
a memory coupled to the camera for storing data corresponding to the image of the object; and
a programmable hardware element coupled to the memory that is configured to implement an image processing function, wherein the programmable hardware element in the smart camera is configured utilizing a hardware description generated from a graphical program, wherein the programmable hardware element implements a hardware implementation of the graphical program, and wherein the programmable hardware element in the smart camera is operable to perform an image processing function on the image;
wherein the programmable hardware element comprises a field programmable gate array (FPGA).
146. The smart camera of claim 145,
wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program.
147. A smart camera, comprising:
a camera for receiving an image of an object;
a memory coupled to the camera for storing data corresponding to the image of the object;
a programmable hardware element coupled to the memory that is configured to implement an image processing function, wherein the programmable hardware element in the smart camera is configured utilizing a hardware description generated from a graphical program, wherein the programmable hardware element implements a hardware implementation of the graphical program, and wherein the programmable hardware element in the smart camera is operable to perform an image processing function on the image; and
a non-volatile memory coupled to the programmable hardware element;
wherein the non-volatile memory is operable to store the hardware description; and
wherein the non-volatile memory is further operable to transfer the hardware description to the programmable hardware element to configure the programmable hardware element.
148. The smart camera of claim 147,
wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program.
149. A method for configuring a smart camera to perform an image processing function, the method comprising:
coupling the smart camera to a computer system, wherein the smart camera comprises at least one functional unit, wherein the functional unit is a programmable hardware element, wherein the computer system stores a graphical program, wherein the graphical program implements the image processing function; and
deploying the graphical program onto the functional unit in the smart camera to configure the functional unit, wherein said deploying the graphical program onto the functional unit comprises:
downloading a hardware configuration onto the programmable hardware element in the smart camera to configure the programmable hardware element, wherein the hardware configuration corresponds to a hardware implementation of the graphical program;
wherein after said downloading the programmable hardware element implements a hardware implementation of the graphical program; and
wherein after said deploying the functional unit is operable to implement the graphical program.
150. The method of claim 149, further comprising:
creating the graphical program prior to said deploying, wherein the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program.
151. The method of claim 149, further comprising:
disconnecting the smart camera from the computer system after said deploying.
Description
FIELD OF THE INVENTION
The present invention relates to graphical programming, and in particular to a system and method for deploying a graphical program onto an image acquisition device. The present invention further relates to a system and method for configuring an image acquisition or machine vision device to perform image processing functions, wherein the instrument or device includes a programmable hardware element and/or a processor and memory.
DESCRIPTION OF THE RELATED ART
Traditionally, high level text-based programming languages have been used by programmers in writing applications programs. Many different high level programming languages exist, including BASIC, C, FORTRAN, Pascal, COBOL, ADA, APL, etc. Programs written in these high level languages are translated to the machine language level by translators known as compilers or interpreters. The high level programming languages in this level, as well as the assembly language level, are referred to as text-based programming environments.
Increasingly computers are required to be used and programmed by those who are not highly trained in computer programming techniques. When traditional text-based programming environments are used, the user's programming skills and ability to interact with the computer system often become a limiting factor in the achievement of optimal utilization of the computer system.
There are numerous subtle complexities which a user must master before he can efficiently program a computer system in a text-based environment. The task of programming a computer system to model or implement a process often is further complicated by the fact that a sequence of mathematical formulas, mathematical steps or other procedures customarily used to conceptually model a process often does not closely correspond to the traditional text-based programming techniques used to program a computer system to model such a process. In other words, the requirement that a user program in a text-based programming environment places a level of abstraction between the user's conceptualization of the solution and the implementation of a method that accomplishes this solution in a computer program. Thus, a user often must substantially master different skills in order to both conceptually model a system and then to program a computer to model that system. Since a user often is not fully proficient in techniques for programming a computer system in a text-based environment to implement his model, the efficiency with which the computer system can be utilized to perform such modeling often is reduced.
Examples of fields in which computer systems are employed to interact with physical systems are the fields of instrumentation, process control, industrial automation, and simulation. Computer measurement and control of devices such as instruments or industrial automation hardware has become increasingly desirable in view of the increasing complexity and variety of instruments and devices available for use. However, due to the wide variety of possible testing and control situations and environments, and also the wide array of instruments or devices available, it is often necessary for a user to develop a custom program to control a desired system.
As discussed above, computer programs used to control such systems traditionally had to be written in text-based programming languages such as, for example, assembly language, C, FORTRAN, BASIC, etc. Traditional users of these systems, however, often were not highly trained in programming techniques and, in addition, text-based programming languages were not sufficiently intuitive to allow users to use these languages without training. Therefore, implementation of such systems frequently required the involvement of a programmer to write software for control and analysis of instrumentation or industrial automation data. Thus, development and maintenance of the software elements in these systems often proved to be difficult.
U.S. Pat. Nos. 4,901,221; 4,914,568; 5,291,587; 5,301,301; and 5,301,336; among others, to Kodosky et al disclose a graphical system and method for modeling a process, i.e., a graphical programming environment which enables a user to easily and intuitively model a process. The graphical programming environment disclosed in Kodosky et al can be considered a higher and more intuitive way in which to interact with a computer. A graphically based programming environment can be represented at a level above text-based high level programming languages such as C, Basic, Java, etc.
The method disclosed in Kodosky et al allows a user to construct a diagram using a block diagram editor. The block diagram may include a plurality of interconnected icons such that the diagram created graphically displays a procedure or method for accomplishing a certain result, such as manipulating one or more input variables and/or producing one or more output variables. In response to the user constructing a diagram or graphical program using the block diagram editor, data structures may be automatically constructed which characterize an execution procedure that corresponds to the displayed procedure. The graphical program may be compiled or interpreted by a computer.
Therefore, Kodosky et al teaches a graphical programming environment wherein a user places or manipulates icons and interconnects or "wires up" the icons in a block diagram using a block diagram editor to create a graphical "program." A graphical program for performing an instrumentation, measurement or automation function, such as measuring a Unit Under Test (UUT) or device, controlling or modeling instruments, controlling or measuring a system or process, or for modeling or simulating devices, may be referred to as a virtual instrument (VI). Thus, a user can create a computer program solely by using a graphically based programming environment. This graphically based programming environment may be used for creating virtual instrumentation systems, modeling processes, control, simulation, and numerical analysis, as well as for any type of general programming.
A graphical program may have a graphical user interface. For example, in creating a graphical program, a user may create a front panel or user interface panel. The front panel may include various graphical user interface elements or front panel objects, such as user interface controls and/or indicators, that represent or display the respective input and/or output that will be used by the graphical program or VI, and may include other icons which represent devices being controlled. The front panel may be comprised in a single window of user interface elements, or may comprise a plurality of individual windows each having one or more user interface elements, wherein the individual windows may optionally be tiled together. When the controls and indicators are created in the front panel, corresponding icons or terminals may be automatically created in the block diagram by the block diagram editor. Alternatively, the user can place terminal icons in the block diagram which may cause the display of corresponding front panel objects in the front panel, either at edit time or later at run time. As another example, the front panel may comprise front panel objects, e.g., the GUI, embedded in the block diagram.
During creation of the block diagram portion of the graphical program, the user may select various function nodes or icons that accomplish his desired result and connect the function nodes together. For example, the function nodes may be connected in one or more of a data flow, control flow, and/or execution flow format. The function nodes may also be connected in a "signal flow" format, which is a subset of data flow. The function nodes may be connected between the terminals of the various user interface elements, e.g., between the respective controls and indicators. Thus the user may create or assemble a graphical program, referred to as a block diagram, graphically representing the desired process. The assembled graphical program may be represented in the memory of the computer system as data structures and/or program instructions. The assembled graphical program, i.e., these data structures, may then be compiled or interpreted to produce machine language that accomplishes the desired method or process as shown in the block diagram.
Input data to a graphical program may be received from any of various sources, such as from a device, unit under test, a process being measured or controlled, another computer program, or from a file. Also, a user may input data to a graphical program or virtual instrument using a graphical user interface, e.g., a front panel as described above. The input data may propagate through the data flow block diagram or graphical program and appear as changes on the output indicators. In an instrumentation application, the front panel can be analogized to the front panel of an instrument. In an industrial automation application the front panel can be analogized to the MMI (Man Machine Interface) of a device. The user may adjust the controls on the front panel to affect the input and view the output on the respective indicators. Alternatively, the front panel may be used merely to view the input and output, or just the output, and the input may not be interactively manipulable by the user during program execution.
Thus, graphical programming has become a powerful tool available to programmers. Graphical programming environments such as the National Instruments LabVIEW product have become very popular. Tools such as LabVIEW have greatly increased the productivity of programmers, and increasing numbers of programmers are using graphical programming environments to develop their software applications. In particular, graphical programming tools are being used for test and measurement, data acquisition, process control, man machine interface (MMI), supervisory control and data acquisition (SCADA) applications, simulation, image processing/machine vision applications, and motion control, among others.
A primary goal of graphical programming, including virtual instrumentation, is to provide the user the maximum amount of flexibility to create his/her own applications and/or define his/her own instrument functionality. In this regard, it is desirable to extend the level at which the user is able to program a device, e.g., extend the level at which a user of instrumentation or industrial automation hardware is able to program an instrument. The evolution of the levels at which the user has been able to program an instrument is essentially as follows. - 1. User level software (LabVIEW, LabWindows CVI, Visual Basic, etc.)
- 2. Kernel level software
- 3. Auxiliary kernel level software (a second kernel running along side the main OS, e.g., InTime, VentureCom, etc.)
- 4. Embedded kernel level software
- 5. Hardware level software (FPGA)
In general, going down the above list, the user is able to create software applications which provide a more deterministic real-time response. Currently, some programming development tools for instrumentation or industrial automation provide an interface at level 1 above. In general, most users are unable and/or not allowed to program at the kernel level or auxiliary kernel level.
It would be highly desirable to provide the user with the ability to develop user level software which operates at the embedded kernel level and/or the hardware level. More particularly, it would be desirable to provide the user with the ability to develop high level software, such as graphical programs, which can then be readily deployed onto an image acquisition device, such as a smart camera. This would provide the user with the dual benefits of being able to program device functionality at the highest level possible (e.g., graphical programs), while also providing the ability to have the created program operate directly on an embedded processor or in hardware for increased speed and efficiency.
SUMMARY OF THE INVENTION
One embodiment of the present invention comprises a computer-implemented system and method for deploying a graphical program onto an image acquisition device, such as an image acquisition board coupled to or included in a computer system, or a smart camera. This provides the user the ability to develop or define desired functionality using graphical programming techniques, while enabling the resulting program to operate directly on an image acquisition device, such as a smart camera. The computer-implemented system and method may be used for configuring an image acquisition device or smart camera instrument to perform an image processing or machine vision function, where the image acquisition device (e.g., smart camera) includes at least one functional unit, such as a processor and memory, or a programmable hardware element.
The user may first create a graphical program which performs or represents the desired functionality. The graphical program may comprise a block diagram which includes a plurality of interconnected nodes which visually indicate functionality of the graphical program. The plurality of nodes may be interconnected in one or more of a data flow, control flow, or execution flow format. The graphical program may include a single diagram or a hierarchy of subprograms or sub-diagrams. In one embodiment, the user may place various constructs in portions of the graphical program to aid in conversion of these portions into hardware form. As the user creates or assembles the graphical program on the display, data structures and/or software code may be automatically created and stored in memory corresponding to the graphical program being created. In an image acquisition or analysis application, the graphical program may implement an image processing function. More generally, in a measurement or instrumentation application, the graphical program may implement a measurement function. A portion of the graphical program may also implement a user interface, described below.
Thus, in one embodiment, the graphical program may be created or assembled by the user arranging on a display a plurality of nodes or icons and then interconnecting the nodes to create the graphical program. In response to the user assembling the graphical program, data structures (and/or program instructions) may be created and stored which represent the graphical program. The graphical program may thus comprise a plurality of interconnected nodes or icons which visually indicates the functionality of the program. As noted above, the graphical program may comprise a block diagram and may also include a user interface portion or front panel portion. Where the graphical program includes a user interface portion, the user may assemble the user interface on the display.
In one embodiment, the graphical program may be create on or stored on a computer system. The method may comprise coupling the image acquisition device to the computer system which stores the graphical program, and, where the image acquisition device includes a functional unit, deploying the graphical program onto the functional unit in the image acquisition device to configure the functional unit. After the deployment of the graphical program, the functional unit may be operable to implement the graphical program. In one embodiment, the image acquisition device may be disconnected from the computer system after deploying the graphical program. The image acquisition device may also be coupled to the computer system over a network, and deploying the graphical program may include the computer system deploying the graphical program over the network to the image acquisition device.
In one embodiment, a computer-implemented method for configuring an image acquisition device (or smart camera) to perform an image processing function may include creating a graphical program on a computer system, as described above, where the graphical program implements the image processing function, and where the graphical program comprises a plurality of interconnected nodes which visually indicate functionality of the graphical program; deploying the graphical program on the image acquisition device, where the image acquisition device is coupled to or comprised in the computer system; the image acquisition device acquiring an image of an object; and the image acquisition device executing to perform the image processing function on the image.
In one embodiment, the image acquisition device may include a processor and a memory, and deploying the graphical program on the image acquisition device may include transferring the graphical program to the memory on the image acquisition device, where the image acquisition device executing to perform the image processing function on the image includes the processor in the image acquisition device executing the graphical program from the memory. In one embodiment, deploying the graphical program on the image acquisition device may include generating an executable program, such as a machine language program, based on the graphical program, which implements at least a portion of the functionality of the graphical program; and transferring the executable program to the memory on the image acquisition device. In one embodiment, compiling the graphical program into an executable program may comprise converting the graphical program first into a text-language program, and then compiling the text-language program into the executable program. The image acquisition device executing to perform the image processing function on the image may include the processor in the image acquisition device executing the executable program from the memory.
In one embodiment, the image acquisition device may include a programmable hardware element, and deploying the graphical program on the image acquisition device may include generating a hardware description based on the graphical program, which describes a hardware implementation of the graphical program, and configuring the programmable hardware element in the image acquisition device utilizing the hardware description. After being configured with the hardware description the programmable hardware element may implement a hardware implementation of the graphical program. The image acquisition device executing to perform the image processing function on the image may include the programmable hardware element in the image acquisition device executing to perform the image processing function on the image.
In one embodiment, the image acquisition device may include a processor and a memory and a programmable hardware element. In this embodiment, deploying the graphical program on the image acquisition device may include transferring a first portion of the graphical program to the memory on the image acquisition device; generating a hardware description based on a second portion of the graphical program, which describes a hardware implementation of the graphical program; and configuring the programmable hardware element in the image acquisition device utilizing the hardware description. After being configured, the programmable hardware element may implement a hardware implementation of the second portion of the graphical program. The image acquisition device executing to perform the image processing function on the image may include the processor in the image acquisition device executing the first portion of the graphical program from the memory and the programmable hardware element executing the second portion of the graphical program.
The first and second portions of the graphical program may each include image processing portions. In some embodiments, the first portion of the graphical program may include an image processing portion, and the second portion of the graphical program may include a control portion. In other embodiments, the first portion of the graphical program may include the control portion, and the second portion of the graphical program may include the image processing portion.
In one embodiment, deploying the graphical program on the image acquisition device may include generating an executable program based on a first portion of the graphical program, and which implements functionality of the graphical program; transferring the executable program to the memory on the image acquisition device; generating a hardware description based on a second portion of the graphical program, where the hardware description describes a hardware implementation of the graphical program; and configuring the programmable hardware element in the image acquisition device utilizing the hardware description, where after being configured, the programmable hardware element implements a hardware implementation of the second portion of the graphical program. The image acquisition device executing to perform the image processing function on the image may include the processor in the image acquisition device executing the executable program from the memory and the programmable hardware element executing the second portion of the graphical program.
In one embodiment, the image acquisition device may include a first processor, a second processor, and at least one memory, and the image acquisition device executing to perform the image processing function on the image may include the first processor in the image acquisition device executing a first portion of the graphical program from the at least one memory and the second processor in the image acquisition device executing a second portion of the graphical program from the at least one memory. For example, the first portion of the graphical program may include a first portion of the image processing function, and the second portion of the graphical program may include a second portion of the image processing function.
During execution, a camera associated with the image acquisition device, or comprised in the smart camera, may acquire an image of an object. For example, the image acquisition device or smart camera may be used in a machine vision inspection application, or a manufacturing assembly application. The at least one functional unit in the image acquisition device or smart camera may then receive the acquired image of the object. The at least one functional unit in the image acquisition device or smart camera may then execute to perform the image processing function (or machine vision function) on the image. The results of this image processing may then be used to make a decision in a machine vision inspection application or may be used to perform an operation in a manufacturing assembly application. During execution of the at least one functional unit in the image acquisition device or smart camera, the user interface portion may be presented on a display. A user or operator may use the displayed user interface portion to view or control the image processing function (or machine vision function).
During operation, the image acquisition device may receive an image present signal, where the image acquisition device acquires the image of the object in response to receiving the image present signal. In another embodiment, the image acquisition device may perform a control operation after executing to perform the image processing function on the image. In yet another embodiment, the image acquisition device may generate a pass/fail indication after executing to perform the image processing function on the image. Alternatively, the image acquisition device may determine characteristics of the image after performing the image processing function, and then perform an operation based on the determined characteristics of the image.
In one embodiment, the image acquisition device may be coupled to a camera, and the image acquisition device acquiring an image of an object may include the camera acquiring the image of the object and the image acquisition device receiving and storing the image. Where the image acquisition device is a smart camera, the smart camera incorporates a camera as well as at least one functional unit.
In one embodiment the graphical program includes a user interface portion, and the method may include presenting the user interface portion on a display and/or receiving user input to the user interface portion on the display to control the image acquisition device, during the functional unit in the image acquisition device executing to perform the image processing function on the image. For example, the user interface portion may operate as a front panel for the image acquisition device.
In one embodiment, the method may further include compiling the user interface portion into executable code for execution by a processor and storing the executable code in a memory, and the processor executing the executable code from the memory to present the user interface portion on the display. The processor and memory may be included on the image acquisition device, and the processor in the image acquisition device may execute the executable code from the memory to present the user interface portion on the display during the image acquisition device executing to perform the image processing function on the image. In another embodiment, the processor and the memory may be included on the computer system, and the computer system may execute the executable code from the memory to present the user interface portion on the display during the image acquisition device executing to perform the image processing function on the image.
Thus the method may operate to configure an image acquisition device, such as an image acquisition board or smart camera, with a graphical program to perform image processing or machine vision functions, where the device includes a functional unit, such as a programmable hardware element or processor/memory.
BRIEF DESCRIPTION OF THE DRAWINGS
A better understanding of the present invention can be obtained when the following detailed description of the preferred embodiment is considered in conjunction with the following drawings, in which:
FIG. 1 illustrates an image acquisition/machine vision system according to one embodiment of the invention;
FIG. 2A illustrates a computer system coupled to a smart camera according to one embodiment of the invention;
FIG. 2B illustrates a smart camera, according to one embodiment of the invention;
FIGS. 3A and 3B illustrate image processing/machine vision systems according to embodiments of the invention;
FIG. 4 is a block diagram of the computer system of FIGS. 1, 2A, and 3A;
FIGS. 5A, 5B and 5C are block diagrams illustrating an interface card or device configured with a programmable hardware element and/or a processor and memory, according to various embodiments of the present invention;
FIGS. 6A-6C are block diagrams illustrating a smart camera configured with a programmable hardware element and/or a processor and memory, according to various embodiments of the invention;
FIG. 7 is a flowchart diagram illustrating deployment of a graphical program on a device, according to one embodiment of the invention;
FIG. 8A is a flowchart illustrating configuration of an image acquisition device or smart camera including a programmable hardware element, according to one embodiment of the invention;
FIG. 8B is a flowchart illustrating configuration of an image acquisition device or smart camera including a processor and memory, according to one embodiment of the invention;
FIG. 9 illustrates the conversion of a graphical program to hardware and software implementations;
FIG. 10 illustrates the generation of various types of hardware and software descriptions from a VDiagram tree;
FIG. 11 illustrates the conversion of a graphical program into a hardware description and the use of the hardware description to program an FPGA;
FIG. 12 illustrates the conversion of a graphical program into a software source code description and the compilation and linkage of the source code;
FIG. 13 is a flowchart diagram illustrating operation of one embodiment of the invention, including compiling a first portion of the graphical program into machine language and converting a second portion of the graphical program into a hardware implementation; and
FIG. 14 is a flowchart diagram illustrating creation of a graphical program according to the preferred embodiment.
While the invention is susceptible to various modifications and alternative forms specific embodiments are shown by way of example in the drawings and will herein be described in detail. It should be understood however, that drawings and detailed description thereto are not intended to limit the invention to the particular form disclosed. But on the contrary the invention is to cover all modifications, equivalents and alternative following within the spirit and scope of the present invention as defined by the appended claims.
DETAILED DESCRIPTION OF THE EMBODIMENTS
Incorporation by Reference
The following U.S. Patents and patent applications are hereby incorporated by reference in their entirety as though fully and completely set forth herein.
U.S. Pat. No. 4,914,568 titled "Graphical System for Modeling a Process and Associated Method," issued on Apr. 3, 1990.
U.S. Pat. No. 6,219,628 titled "System and Method for Configuring an Instrument to Perform Measurement Functions Utilizing Conversion of Graphical Programs into Hardware Implementations".
U.S. Pat. No. 6,173,438 titled "Embedded Graphical Programming System" filed Aug. 18, 1997, whose inventors are Jeffrey L. Kodosky, Darshan Shah, Samson DeKey, and Steve Rogers.
U.S. patent application Ser. No. 09/499,503 titled "System and Method for Configuring a Programmable Hardware Instrument to Perform Measurement Functions Utilizing Estimation of the Hardware Implementation and Management of Hardware Resources", filed on Feb. 7, 2000, whose inventors are Jeffrey L. Kodosky, Hugo Andrade, Brian Keith Odom, Cary Paul Butler, and Andrew Mihal.
U.S. patent application Ser. No. 09/891,571 titled "System and Method for Configuring an Instrument to Perform Measurement Functions Utilizing Conversion of Graphical Programs into Hardware Implementations" filed on Jun. 25, 2001, whose inventors are Jeffrey L. Kodosky, Hugo Andrade, Brian Keith Odom, Cary Paul Butler, and Kevin L. Schultz.
U.S. patent application Ser. No. 09/745,023 titled "System and Method for Programmatically Generating a Graphical Program in Response to Program Information," filed Dec. 20, 2000, whose inventors are Ram Kudukoli, Robert Dye, Paul F. Austin, Lothar Wenzel and Jeffrey L. Kodosky.
U.S. patent application Ser. No. 09/595,003 titled "System and Method for Automatically Generating a Graphical Program to Implement a Prototype", filed Jun. 13, 2000, whose inventors are Nicolas Vazquez, Jeffrey L. Kodosky, Ram Kudukoli, Kevin L. Schultz, Dinesh Nair, and Christophe Caltagirone.
U.S. patent application Ser. No. 09/587,682 titled "System and Method for Automatically Generating a Graphical Program to Perform an Image Processing Algorithm", filed on Jun. 5, 2000, whose inventors are Nicolas Vazquez, Jeffrey L. Kodosky, Ram Kudukoli, Kevin L. Schultz, Dinesh Nair and Christophe Caltagirone.
The LabVIEW and BridgeVIEW graphical programming manuals, including the "G Programming Reference Manual", available from National Instruments Corporation, are also hereby incorporated by reference in their entirety. ps FIG. 1—Image Acquisition or Machine Vision System
FIG. 1 illustrates a host computer system 102 coupled to an image acquisition device 134. As used herein, the term "image acquisition device" is intended to include any of various types of devices that are operable to acquire and/or store an image. An image acquisition device may also optionally be further operable to analyze or process the acquired or stored image. Examples of an image acquisition device include an image acquisition (or machine vision) card (also called a video capture board), a device external to a computer that operates similarly to an image acquisition card, a smart camera, a robot having machine vision, and other similar types of devices.
As used herein, the terms "image processing" and "machine vision" are used interchangeably to refer to the processing of images to extract useful information from the image or determine characteristics of the image (or to determine characteristics of one or more objects displayed in the image). The term "image processing" is used herein to refer to both "image processing" and "machine vision", to the extent these terms have different meanings. The term "image processing function" includes tools such as edge detection, blob analysis, pattern matching, and other image processing functions. The term "image processing function" may also include an operation or decision that is performed in response to the information extracted or characteristics determined from the image. The term "image processing function" is also intended to include an image processing (or machine vision) algorithm that combines a sequence of two or more image processing functions or tools and/or decision operations that process an image in a desired way or which implement an image processing or machine vision application, such as part inspection, automated assembly, image analysis, pattern matching, edge detection, etc.
Thus, FIG. 1 illustrates an exemplary image acquisition or machine vision system 100. As FIG. 1 shows, the image acquisition device 134 may in turn couple to or comprise a camera 132. The image acquisition device 134 may include a functional unit for performing an image processing function as described below.
The host computer 102 may comprise a CPU, a display screen, memory, and one or more input devices such as a mouse or keyboard as shown. The computer 102 may operate with the image acquisition device to analyze, measure or control a device or process 150. Alternatively, the computer 102 may be used only to configure a functional unit in the image acquisition device.
As used herein, the term "functional unit" may include a processor and memory or a programmable hardware element. The term "functional unit" may include one or more processors and memories and/or one or more programmable hardware elements. As used herein, the term "processor" is intended to include any of types of processors, CPUs, microcontrollers, or other devices capable of executing software instructions. As used herein, the term "programmable hardware element" is intended to include various types of programmable hardware, reconfigurable hardware, programmable logic, or field-programmable devices (FPDs), such as one or more FPGAs (Field Programmable Gate Arrays), or one or more PLDs (Programmable Logic Devices), such as one or more Simple PLDs (SPLDs) or one or more Complex PLDs (CPLDs), or other types of programmable hardware.
As shown, video device or camera 132 may be coupled to the computer 102 via the image acquisition device or card 134. The camera 132 and/or image acquisition device 134 may couple to the computer 102 through a serial bus, a network, or through other means.
The image acquisition system 100 may be used in an manufacturing assembly, test, measurement, and/or control application, among others. For illustration purposes, a unit under test (UUT) 150 is shown which may be positioned by a motion control device 136 (and interface card 138), and imaged and analyzed by the camera 132 and image acquisition device 134. It is noted that in various other embodiments the UUT 150 may comprise a process or system to be measured and/or analyzed.
Referring again to FIG. 1, the computer 102 may include a memory medium on which computer programs, e.g., graphical programs, according to the present invention may be stored. As used herein, the term "memory medium" includes a non-volatile medium, e.g., a magnetic media or hard disk, or optical storage; a volatile medium, such as computer system memory, e.g., random access memory (RAM) such as DRAM, SRAM, EDO RAM, RAMBUS RAM, DR DRAM, etc.; or an installation medium, such as a CD-ROM or floppy disks 104, on which the computer programs according to the present invention may be stored for loading into the computer system. The term "memory medium" may also include other types of memory or combinations thereof.
The memory medium may be comprised in the computer 102 where the programs are executed or may be located on a second computer which is coupled to the computer 102 through a network, such as a local area network (LAN), a wide area network (WAN), or the Internet. In this instance, the second computer operates to provide the program instructions through the network to the computer 102 for execution.
The software programs of the present invention may be stored in a memory medium of the respective computer 102, or in a memory medium of another computer, and executed by the CPU. The CPU executing code and data from the memory medium thus comprises a means for deploying a graphical program onto an image acquisition device, e.g., a smart camera, according to the steps described below.
The memory medium may store a graphical programming development system for developing graphical programs. The memory medium may also store one or more computer programs which are executable to deploy a graphical program, such as by converting at least a portion of a graphical program into a form for configuring or programming a programmable hardware element, by executing the graphical program natively on a processor, or by converting the graphical program to a different form for execution by a processor and memory. The image acquisition device 134 in FIG. 1 may be controlled by or configured by graphical software programs which are deployed or downloaded to the functional unit on the device 134.
As described below, in one embodiment the graphical program may be deployed by either one or more of: 1) converting the graphical program (or a portion thereof) into a hardware implementation and configuring the programmable hardware element with this hardware implementation, 2) transferring the graphical program (or a portion thereof) to a memory of the functional unit for execution by a processor (wherein the processor may execute a graphical program execution engine and optionally a real time operating system), or 3) compiling the graphical program (or a portion thereof) into an executable program and transferring the executable program to a memory of the functional unit for execution by a processor (wherein the processor may optionally execute a real time operating system).
In the present application, the term "graphical program" or "block diagram" is intended to include a program comprising graphical code, e.g., two or more nodes or icons interconnected in one or more of a data flow, control flow, or execution flow format, where the interconnected nodes or icons may visually indicates the functionality of the program. Thus the terms "graphical program" or "block diagram" are each intended to include a program comprising a plurality of interconnected nodes or icons which visually indicates the functionality of the program. A graphical program may comprise a block diagram and may also include a user interface portion or front panel portion. The user interfac |
