Data retrieval system using secondary information of primary data to be retrieved as retrieval key

Abstract

A data retrieval system capable of carrying out the retrieval using the secondary information of the primary data to be retrieved such as environmental information or structural information as the retrieval key. In the system, the primary data which are filing target to be stored and retrieved are stored in a first memory while secondary information associated with the primary data, which are not directly related with data content of the primary data, are stored in a second memory in correspondence to the primary data stored in the first memory. Then, when a retrieval key specifying a desired retrieval of the primary data from the first memory is entered, the secondary information is selected from the second memory in accordance with the retrieval key, and the primary data corresponding to the secondary information selected in accordance with the retrieval key is retrieved and outputted as a retrieval result.

Claims

What is claimed is:

1. A data retrieval system, comprising:

first memory means for storing primary data which are a filing target to be stored in the system and retrieved from the system;

analysis means for automatically acquiring secondary information at a time of manipulation or input of the primary data, the secondary information being environmental information indicating an environmental condition of the primary data which is not directly indicative of the data content of the primary data;

second memory means for storing the secondary information associated with the primary data, in correspondence to the primary data stored in the first memory means;

input means for specifying a retrieval key to be used in selecting the secondary information associated with a desired primary data, said retrieval key not specifying any content of the desired primary data;

retrieval means for selecting the secondary information from the second memory means in accordance with the retrieval key entered by the input means, and then identifying for retrieval, based on the secondary information, the primary data stored in correspondence to the secondary information selected in accordance with the retrieval key; and

output means for outputting the primary data identified for retrieval by the retrieval means as a retrieval result for the desired primary data.

2. The system of claim 1, wherein the secondary information is an environmental information indicating an environmental condition of the primary data.

3. The system of claim 2, further comprising means for obtaining a position information indicating a position at which the primary data had been manipulated, and wherein the environmental information is the position information.

4. The system of claim 2, further comprising schedule management means for managing a schedule of a user, and wherein the environmental information is a schedule information indicating the schedule of a user at a time of manipulation of the primary data.

5. The system of claim 2, further comprising imaging means for taking image information at a time of manipulation of the primary data, and wherein the environmental information is the image information taken at a time of manipulation of the primary data.

6. The system of claim 2, further comprising means for measuring a time information indicating start time and end time of manipulation of the primary data, and wherein the environmental information is the time information.

7. The system of claim 2, further comprising means for measuring a time information indicating start time and end time of manipulation of the primary data, and wherein the environmental information is an urgency level of the manipulation of the primary data which is determined from the time information.

8. The system of claim 2, further comprising means for measuring a time information indicating start time and end time of manipulation of the primary data, and wherein the environmental information is a number of times for which the manipulation of the primary data had taken place which is determined from the time information.

9. The system of claim 2, wherein the primary data are speech data, and the environmental information is an acoustic information indicating an acoustic environment in which the speech data had been originally given to the system.

10. The system of claim 9, wherein the acoustic information indicates the acoustic environment concerning speakers who had generated the speech data.

11. The system of claim 9, wherein the acoustic information indicates the acoustic environment concerning a location at which the speech data had been generated.

12. The system of claim 9, wherein the acoustic information indicates the acoustic environment concerning a surrounding sound in which the speech had been generated.

13. The system of claim 2, wherein the environmental information is a device information indicating a computer device by which the primary data had been manipulated.

14. The system of claim 13, wherein the device information identifies the computer device itself.

15. The system of claim 13, wherein the device information indicates a location of the computer device.

16. The system of claim 1, wherein the secondary information indicates a characteristic external appearance of the primary data.

17. The system of claim 16, wherein the characteristic external appearance of the primary document data is specified in terms of features of a medium on which the primary document data had been originally given to the system.

18. The system of claim 16, wherein the characteristic external appearance of the primary document data is specified in terms of visual features of an image of the primary document data by which the primary document data had been originally given to the system.

19. The system of claim 16, wherein the characteristic external appearance of the primary document data is specified in terms of a layout of the primary document data in which the primary document data had been originally given to the system.

20. A data retrieval system, comprising:

first memory means for storing primary document data which are a filing target to be stored in the system and retrieved from the system;

analysis means for automatically acquiring secondary information at a time of input of the primary document data, the secondary information indicating a characteristic external appearance of the primary document data which is not directly indicative of the data content of the primary document data;

second memory means for storing the secondary information associated with the primary document data, in correspondence to the primary document data stored in the first memory means;

input means for specifying a retrieval key to be used in selecting the secondary information associated with a desired primary document data, said retrieval key not specifying any content of the desired primary document data;

retrieval means for selecting the secondary information from the second memory means in accordance with the retrieval key entered by the input means, and then retrieving, based on the secondary information, the primary document data stored in correspondence to the secondary information selected in accordance with the retrieval key; and

output means for outputting the primary document data retrieved by the retrieval means as a retrieval result for the desired primary document data.

21. The system of claim 1, wherein the retrieval key entered by the input means specifies secondary information.

22. The system of claim 21, wherein the secondary information includes one of position information, schedule information, image information, time information, acoustic information, device information, and characteristic external appearance information.

23. A data retrieval system, comprising:

first memory means for storing primary data which are a filing target to be stored in the system and retrieved from the system;

analysis means for automatically acquiring secondary information at a time of manipulation or input of the primary data, the secondary information being environmental information indicating an environmental condition of the primary data which is not directly indicative of the data content of the primary data;

second memory means for storing the secondary information associated with the primary data, in correspondence to the primary data stored in the first memory means;

input means for entering a retrieval key specifying at least one of the secondary information associated with a desired primary data, said retrieval key not specifying any content of the desired primary data;

retrieval means for retrieving, based on the secondary information, the primary data stored in correspondence to the secondary information specified by the retrieval key; and

output means for outputting the primary data retrieved by the retrieval means as a retrieval result for the desired primary data.

24. The system of claim 23, wherein the secondary information includes one of position information, schedule information, image information, time information, acoustic information, device information, and characteristic external appearance information.

25. A data retrieval system, comprising:

first memory means for storing time series data which are a filing target to be stored in the system and retrieved from the system;

structural analysis means for extracting from the time series data key data indicative of a semantic content of the time series data, and generating a structural information containing the key data and a link information for linking the key data with the time series data;

second memory means for storing the structural information generated by the structural analysis means;

input means for specifying a retrieval key indicating the key data associated with a desired time series data to be retrieved, said retrieval key not specifying any content of the desired time series data;

retrieval means for retrieving the structural information which contains the key data indicated by the retrieval key from the second memory means, and identifying for retrieval the time series data identified by the link information contained in the structural information retrieved in accordance with the key data; and

output means for outputting the time series data identified for retrieval by the retrieval means as a retrieval result for the desired time series data.

26. The system of claim 25, wherein the retrieval means retrieves only a portion of the desired time series data which is linked with the key data used as the retrieval key according to the link information contained in the structural information retrieved in accordance with the key data.

27. The system of claim 25, wherein the retrieval means retrieves only the time series data which are linked with the key data used as the retrieval key according to the link information contained in the structural information retrieved in accordance with the key data.

28. The system of claim 25, wherein the structural information further contains an environmental information indicating an environmental condition of the time series data, the retrieval means also retrieves the environmental information of the time series data which are linked with the key data used as the retrieval key, and the output means also visually presents the environmental information.

29. The system of claim 25, wherein the input means also enters a new key data specified by a user, the structural analysis means generating a new structural information from the new key data.

30. The system of claim 25, wherein the input means also enters a new structural information specified by a user, to be stored in the second memory means.

31. A method of data retrieval, comprising the steps of:

storing primary data which are a filing target to be stored and retrieved, in a first memory;

automatically acquiring secondary information at a time of manipulation or input of the primary data, the secondary information being environmental information indicating an environmental condition of the primary data which is not directly indicative of the data content of the primary data;

storing the secondary information associated with the primary data, in a second memory in correspondence to the primary data stored in the first memory;

specifying a retrieval key to be used in selecting the secondary information associated with a desired primary data, said retrieval key not specifying any content of the desired primary data;

selecting the secondary information from the second memory in accordance with the retrieval key entered at the entering step, and then identifying for retrieval, based on the secondary information, the primary data stored in correspondence to the secondary information selected in accordance with the retrieval key; and

outputting the primary data identified for retrieval at the identifying step as a retrieval result for the desired primary data.

32. The method of claim 31, wherein the retrieval key specifies secondary information.

33. The method of claim 32, wherein the secondary information includes one of position information, schedule information, image information, time information, acoustic information, device information, and characteristic external appearance information.

34. The method of claim 31, wherein the secondary information is an environmental information indicating an environmental condition of the primary data.

35. The method of claim 34, further comprising the step of obtaining a position information indicating a position at which the primary data had been manipulated, and wherein the environmental information is the position information.

36. The method of claim 34, further comprising the step of managing a schedule of a user, and wherein the environmental information is a schedule information indicating the schedule of a user at a time of manipulation of the primary data.

37. The method of claim 34, further comprising the step of taking image information at a time of manipulation of the primary data, and wherein the environmental information is the image information taken at a time of manipulation of the primary data.

38. The method of claim 34, further comprising the step of measuring a time information indicating start time and end time of manipulation of the primary data, and wherein the environmental information is the time information.

39. The method of claim 34, further comprising the step of measuring a time information indicating start time and end time of manipulation of the primary data, and wherein the environmental information is an urgency level of the manipulation of the primary data which is determined from the time information.

40. The method of claim 34, further comprising the step of measuring a time information indicating start time and end time of manipulation of the primary data, and wherein the environmental information is a number of times for which the manipulation of the primary data had taken place which is determined from the time information.

41. The method of claim 34, wherein the primary data are speech data, and the environmental information is an acoustic information indicating an acoustic environment in which the speech data had been originally given.

42. The method of claim 41, wherein the acoustic information indicates the acoustic environment concerning speakers who had generated the speech data.

43. The method of claim 41, wherein the acoustic information indicates the acoustic environment concerning a location at which the speech data had been generated.

44. The method of claim 41, wherein the acoustic information indicates the acoustic environment concerning a surrounding sound in which the speech had been generated.

45. The method of claim 34, wherein the environmental information is a device information indicating a computer device by which the primary data had been manipulated.

46. The method of claim 45, wherein the device information identifies the computer device itself.

47. The method of claim 45, wherein the device information indicates a location of the computer device.

48. A method of data retrieval, comprising the steps of:

storing primary document data, which are a filing target to be stored and retrieved, in a first memory;

automatically acquiring secondary information at a time of input of the primary document data, the secondary information indicating a characteristic external appearance of the primary document data which is not directly indicative of the data content of the primary document data;

storing the secondary information associated with the primary document data, in a second memory in correspondence to the primary document data stored in the first memory;

specifying a retrieval key to be used in selecting the secondary information associated with a desired primary document data, said retrieval key not specifying any content of the desired primary document data;

selecting the secondary information from the second memory solely in accordance with the retrieval key entered at the entering step, and then retrieving, based on the secondary information, the primary document data stored in correspondence to the secondary information selected in accordance with the retrieval key; and

outputting the primary document data retrieved at the retrieving step as a retrieval result for the desired primary document data.

49. The method of claim 48, wherein the characteristic external appearance of the primary document data is specified in terms of features of a medium on which the primary document data had been originally given.

50. The method of claim 48, wherein the characteristic external appearance of the primary document data is specified in terms of features of a fixing material by which the primary document data had been originally fixed on a medium on which the primary document data had been originally given.

51. The method of claim 48, wherein the characteristic external appearance of the primary document data is specified in terms of visual features of an image of the primary document data by which the primary document data had been originally given.

52. The method of claim 48, wherein the characteristic external appearance of the primary document data is specified in terms of a layout of the primary document data in which the primary document data had been originally given.

53. A method of data retrieval, comprising the steps of:

storing primary data which are a filing target to be stored in the system and retrieved from the system, in a first memory;

automatically acquiring secondary information at a time of manipulation or input of the primary data, the secondary information being environmental information indicating an environmental condition of the primary data which is not directly indicative of the data content of the primary data;

storing the secondary information associated with the primary data, in a second memory in correspondence to the primary data stored in the first memory;

entering a retrieval key specifying at least one of the secondary information associated with a desired primary data, said retrieval key not specifying any content of the desired primary data;

retrieving based on the secondary information, the primary data stored in correspondence to the secondary information specified by the retrieval key; and

outputting the primary data retrieved at the retrieving step as a retrieval result for the desired primary data.

54. A method of data retrieval, comprising the steps of:

storing time series data which are a filing target to be stored and retrieved, in a first memory;

extracting from the time series data key data indicative of a semantic content of the time series data, and generating a structural information containing the key data and a link information for linking the key data with the time series data;

storing the structural information in a second memory;

specifying a retrieval key indicating the key data associated with a desired time series data to be retrieved, said retrieval key not specifying any content of the desired time series data;

retrieving the structural information which contains the key data indicated by the retrieval key from the second memory, and identifying for retrieval the time series data identified by the link information contained in the structural information retrieved in accordance with the key data; and

outputting the time series data identified for retrieval at the identifying step as a retrieval result for the desired time series data.

55. The method of claim 54, wherein the retrieving step retrieves only a portion of the desired time series data which is linked with the key data used as the retrieval key according to the link information contained in the structural information retrieved in accordance with the key data.

56. The method of claim 54, wherein the retrieving step retrieves only the time series data which are linked with the key data used as the retrieval key according to the link information contained in the structural information retrieved in accordance with the key data.

57. The method of claim 54, wherein the structural information further contains an environmental information indicating an environmental condition of the time series data, the retrieving step also retrieves the environmental information of the time series data which are linked with the key data used as the retrieval key, and the outputting step also visually presents the environmental information.

58. The method of claim 54, further comprising the step of entering a new key data specified by a user, the extracting step generating a new structural information from the new key data.

59. The method of claim 54, further comprising the step of entering a new structural information specified by a user, to be stored in the second memory.

60. A data retrieval system, comprising:

first memory means for storing primary document data which are a filing target to be stored in the system and retrieved from the system;

analysis means for automatically acquiring secondary information at a time of input of the primary document data, the secondary information indicating a characteristic external appearance of the primary document data which is not directly indicative of the data content of the primary document data;

second memory means for storing the secondary information associated with the primary document data, in correspondence to the primary document data stored in the first memory means;

input means for entering a retrieval key specifying at least one of the secondary information associated with a desired primary document data, said retrieval key not specifying any content of the desired primary document data;

retrieval means for identifying for retrieval, based on the secondary information, the primary document data stored in correspondence to the secondary information specified by the retrieval key; and

output means for outputting the primary document data identified for retrieval by the retrieval means as a retrieval result for the desired primary document data.

61. A method of data retrieval, comprising the steps of:

storing primary document data, which are a filing target to be stored and retrieved, in a first memory;

automatically acquiring secondary information at a time of input of the primary document data, the secondary information indicating a characteristic external appearance of the primary document data which is not directly indicative of the data content of the primary document data;

storing the secondary information associated with the primary document data, in a second memory in correspondence to the primary document data stored in the first memory;

entering a retrieval key specifying at least one of the secondary information associated with a desired primary document data, said retrieval key not specifying any content of the desired primary document data;

identifying for retrieval, based on the secondary information, the primary document data stored in correspondence to the secondary information specified by the retrieval key; and

outputting the primary document data identified for retrieval at the identifying step as a retrieval result for the desired primary document data.

62. A method of keeping, in a memory unit connected to a CPU, and retrieving from the memory unit, data contained in physical records, said method comprising the steps of:

inputting data contained in a physical record into the memory unit;

automatically acquiring at least one secondary information data about the physical record during the step of inputting, the secondary information data being indicative of the physical conditions of and around the physical record but unrelated to the data contained in the physical record;

storing the secondary information data about the physical record in correspondence to the data contained in the physical record;

entering a retrieval key for specifying at least one secondary information data;

identifying for retrieval the data contained in the physical record stored in correspondence to the at least one secondary information data specified by the retrieval key; and

outputting the data contained in the physical record identified for retrieval at the identifying step as a retrieval result.

63. The method of claim 62, wherein the step of inputting includes the step of inputting one or more of document data, image data, and acoustic data.

64. The method of claim 62, wherein the step of automatically acquiring at least one secondary information data includes the step of automatically aquiring secondary information data that is indicative of environmental conditions during the step of inputting.

65. The method of claim 62, wherein the step of automatically acquiring at least one secondary information data includes the step of automatically aquiring position information data during the step of inputting using a global positioning system.

66. The method of claim 62, wherein the step of automatically acquiring at least one secondary information data includes the step of automatically aquiring environmental conditions, including one or more of temperature, humidity, and pressure, during the step of inputting.

67. The method of claim 62, wherein the step of automatically acquiring at least one secondary information data includes the step of automatically aquiring user health conditions, including one or more of pulse rate, blood pressure, body temperature, and blood sugar level, during the step of inputting.

68. The method of claim 62, wherein the step of automatically acquiring at least one secondary information data includes the step of automatically aquiring secondary information data that is indicative of an external appearance of the physical record.

Description

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a data retrieval system for retrieving desired data such as document data and time series data from a large number of data stored in a database. It is to be noted that the term "detection" is often used in the literature of the field instead of the term "retrieval" used in the following description. The present specification adheres to the use of the term "retrieval" throughout.

2. Description of the Background Art

In the conventional document retrieval system or word processor, in order for the user to retrieve the document that had been retrieved in the past again, it has been necessary for the user to either call up the document by recalling the title or the content of the retrieved document, or retrieve the document by recalling the retrieval key used in the past retrieval. Here, the retrieval key must be specified as a character string, while the title or the content of the document must be specified as a natural language expression indicative or at least suggestive of the title or the content itself. Consequently, in a case the user cannot recall the retrieval key, the title or the content of the document, or in a case the user cannot express it appropriately as the character string or the natural language expression, it has been impossible to carry out the retrieval.

On the other hand, conventionally, there has been an electronic filing system in which the document image is taken by a scanner and stored as the image data, so as to replace the conventional document files in paper. However, such an electronic filing system could have been quite inconvenient especially when an amount of documents stored in such a filing system becomes huge, as it is possible for the user to forget the keyword or retrieval key assigned to a desired document after that document has been filed such that it can be extremely difficult to find out the desired document out of a huge number of filed documents,

In this regard, in a case of the document files in paper, there still remains a clue for finding out the desired document such as the characteristic external appearance of each file. For instance, the user can remember the desired document as one with stains of coffee, such that this desired document can be found out relatively easily by checking out those with stains of coffee alone. However, in a case of the electronic filing system, such an information concerning the characteristic external appearance of each file is going to be disregarded as insignificant at a time of recording, so that it has been impossible in the conventional electronic filing system to carry out the retrieval of the document based on such a natural recollection of each file by the human user.

In particular, in a system for filing the result of processing for each document image such as the document structure analysis or the character recognition, the noises such as stains are going to be eliminated at a time of the processing in order to improve the processing performance. Consequently, in such a filing system, the document to be displayed at a time of the retrieval is only the result of such a processing which may have the external appearance different from that of the original document remembered by the human user, and it has been difficult to judge whether the retrieved document is indeed the desired document or not by a glance.

Now, in recent years, in conjunction with the advance of the processing performance of the computer, it has become possible to carry out the recording or the retrieval of the multi-media data such as speeches, images, etc. in various manners for the purposes of recording, reproducing, and editing such multi-media data.

In addition, as it becomes possible to utilize a large capacity recording medium relatively easily, many applications using input/output, manipulation or editing processing of the multi-media data are developed, but the presently available applications are limited to the realization of the editing function utilizing the straightforward handling of the multi-media data.

As for the recording of the multi-media data which requires a huge amount of data to be stored, it is possible to realize it by using a recording medium having a sufficient recording capacity. However, the retrieval and the output of the recorded multi-media data are not realized at truly satisfactory level, because the work load of the computer for finding out a necessary portion out of huge recorded data becomes increasingly large as an amount of stored data increases.

In particular, in a case of dealing with the time series data, the time required for the retrieval becomes longer in proportion to the length of the time series data itself.

Moreover, the user may wish to retrieve and output the information other than the multi-media data itself, such as the significant portion within the multi-media data, or the key data indicating the meaning represented by that significant portion. Here, the key data cannot be obtained by the conventionally available straightforward recording, reproducing, and editing processing of the multi-media data.

For instance, when the input utterance of "Well, I want one orange juice, no, three orange juices." is given as the speech data, it may be helpful for the user to obtain only the meaningful information indicating that "I want three orange juices." rather than this speech data as it is.

In order to make such a key data available, it is necessary to carry out the recognition processing for the multi-media data to detect the key data, produce a structure information containing at least the detected key data itself and a link data relating the detected key data with the original time series data, and output a combination of a part of the structure information containing the key data as a meaningful portion.

In such a structural analysis processing for generating the structure information automatically such as the recognition processing, there are problems concerning the real time processing and the error in the detected key data.

Of these, the problem of the real time processing is becoming increasingly less problematic as the real time realization of the recognition processing for the speeches and images is becoming plausible due to the advance of the computer performance.

On the other hand, the problem of the error in the structural analysis result can be resolved by providing a provision for producing more accurate key data by the correction of the automatically generated key data or the updating of the key data itself by the user operation, assuming that the error may be contained in the key data.

However, up to now, there has been no system which utilizes such a key data as the retrieval key for the retrieval operation.

SUMMARY OF THE INVENTION

It is therefore an object of the present invention to provide a data retrieval system capable of carrying out the retrieval using the secondary information of the primary data to be retrieved such as environmental information or structural information as the retrieval key.

More specifically, it is an object of the present invention to provide a data retrieval system capable of carrying out the retrieval even when the user cannot recall the retrieval key, the title or the content of the document, or in a case the user cannot express it appropriately as the character string or the natural language expression.

It is another object of the present invention to provide a data retrieval system capable of carrying out the retrieval according to the characteristic external appearance of each file as the retrieval key.

It is another object of the present invention to provide a data retrieval system capable of obtaining only the meaningful information from the multi-media time series data by retrieving and outputting only a significant portion of the stored time series data that can be required by the user.

According to one aspect of the present invention there is provided a data retrieval system, comprising: first memory means for storing primary data which are filing target to be stored in the system and retrieved from the system; second memory means for storing secondary information associated with the primary data which are not directly related with data content of the primary data, the secondary information being stored in correspondence to the primary data stored in the first memory means; input means for entering a retrieval key specifying a desired retrieval of the primary data from the first memory means; retrieval means for selecting the secondary information from the second memory means in accordance with the retrieval key entered by the input means, and retrieving the primary data corresponding to the secondary information selected in accordance with the retrieval key; and output means for outputting the primary data retrieved by the retrieval means as a retrieval result.

According to another aspect of the present invention there is provided a data retrieval system, comprising: first memory means for storing time series data which are filing target to be stored in the system and retrieved from the system; structural analysis means for extracting key data indicative of a semantic content of the time series data, and generating a structural information containing the key data and a link information relating the key data with the time series data; second memory means for storing the structural information generated by the structural analysis means; input means for entering a retrieval key in terms of the key data of a desired time series data to be retrieved; retrieval means for retrieving the structural information from the second memory means in accordance with the retrieval key entered by the input means, and retrieving the desired time series data according to the link information contained in the structural information retrieved in accordance with the key data; and output means for outputting the time series data retrieved by the retrieval means as a retrieval result.

According to another aspect of the present invention there is provided a method of data retrieval, comprising the steps of: storing primary data which are filing target to be stored and retrieved, in a first memory; storing secondary information associated with the primary data which are not directly related with data content of the primary data, the secondary information being stored in a second memory in correspondence to the primary data stored in the first memory; entering a retrieval key specifying a desired retrieval of the primary data from the first memory; selecting the secondary information from the second memory in accordance with the retrieval key entered at the entering step, and retrieving the primary data corresponding to the secondary information selected in accordance with the retrieval key; and outputting the primary data retrieved at the retrieving step as a retrieval result.

According to another aspect of the present invention there is provided a method of data retrieval, comprising the steps of: storing time series data which are filing target to be stored and retrieved, in a first memory extracting key data indicative of a semantic content of the time series data, and generating a structural information containing the key data and a link information relating the key data with the time series data; storing the structural information generated by the structural analysis means in a second memory; entering a retrieval key in terms of the key data of a desired time series data to be retrieved; retrieving the structural information from the second memory in accordance with the retrieval key entered at the entering step, and retrieving the desired time series data according to the link information contained in the structural information retrieved in accordance with the key data; and outputting the time series data retrieved at the retrieving step as a retrieval result.

Other features and advantages of the present invention will become apparent from the following description taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic block diagram of the first general embodiment of the data retrieval system according to the present invention.

FIG. 2 is a table summarizing definitions of various data used in the system of FIG. 1.

FIG. 3 is a schematic block diagram of a retrieval target data input unit in the system of FIG. 1.

FIG. 4 is a schematic block diagram of an environmental analysis target data input unit in the system of FIG. 1.

FIG. 5 is a timing chart showing timings for analysis of the environmental analysis target data used in the system of FIG. 1.

FIG. 6 is a table summarizing exemplary analysis for various environmental analysis target data used in the system of FIG. 1.

FIG. 7 is a schematic block diagram of one modified configuration for the first general embodiment of the data retrieval system according to the present invention.

FIG. 8 is a schematic block diagram of another modified configuration for the first general embodiment of the data retrieval system according to the present invention.

FIG. 9 is a schematic block diagram for a physical configuration of the first specific embodiment of the system of FIG. 1.

FIG. 10 is a schematic block diagram for a functional configuration of the first specific embodiment of the system of FIG. 1.

FIG. 11 is a table showing a format and examples for data in the document storage unit in the configuration of FIG. 10.

FIG. 12 is a flow chart for the operation of the control unit in the configuration of FIG. 10 in a case of the retrieval.

FIG. 13 is a flow chart for the operation of the retrieval unit in the configuration of FIG. 10 in a case of the retrieval.

FIG. 14 is a table showing a format and examples for data in the position and document information storage unit in the configuration of FIG. 10.

FIG. 15 is a flow chart for the operation of the retrieval unit in the configuration of FIG. 10 in a case of the new document production.

FIG. 16 is a flow chart for the operation of the control unit in the configuration of FIG. 10 in a case of the location registration.

FIG. 17 is a flow chart for the operation of the position measurement unit in the configuration of FIG. 10 in a case of the location registration.

FIG. 18 is a table showing a format and examples for data in the location name storage unit in the configuration of FIG. 10.

FIG. 19 is a flow chart for the operation of the control unit in the configuration of FIG. 10 in a case of the location retrieval.

FIG. 20 is a flow chart for the operation of the retrieval result display unit in the configuration of FIG. 10 in a case of the location retrieval.

FIG. 21 is an illustration of an exemplary display by the retrieval result display unit in the configuration of FIG. 10 during the location retrieval.

FIG. 22 is a flow chart for the operation of the retrieval unit in the configuration of FIG. 10 in a case of the location retrieval.

FIG. 23 is a flow chart for the retrieval processing at the retrieval unit in the configuration of FIG. 10 in a case of the location retrieval.

FIG. 24 is an illustration of an exemplary display by the retrieval result display unit in the configuration of FIG. 10 after the location retrieval.

FIG. 25 is a flow chart for the operation of the control unit in the configuration of FIG. 10 in a case urging the location registration.

FIG. 26 is a flow chart for the operation of the retrieval result display unit in the configuration of FIG. 10 in a case urging the location registration.

FIG. 27 is a flow chart for the operation of the retrieval unit in the configuration of FIG. 10 in a case urging the location registration.

FIG. 28 is an illustration of an exemplary display by the retrieval result display unit in the configuration of FIG. 10 during a case of urging the location registration.

FIG. 29 is an illustration of an exemplary display by the retrieval result display unit in the configuration of FIG. 10 after a case of urging the location registration.

FIG. 30 is a schematic block diagram for a functional configuration of the second specific embodiment of the system of FIG. 1.

FIG. 31 is a flow chart for the operation of the retrieval unit in the configuration of FIG. 30 in a case of document loop up or correction.

FIG. 32 is a flow chart for the operation of the retrieval unit in the configuration of FIG. 30 in a case of document production.

FIG. 33 is a schematic block diagram for a functional configuration of a first device in the third specific embodiment of the system of FIG. 1.

FIG. 34 is a schematic block diagram for a functional configuration of a second device in the third specific embodiment of the system of FIG. 1.

FIG. 35 is a flow chart for the operation of the retrieval unit in the first device of FIG. 33 in a case of document look up or correction.

FIG. 36 is a table showing a format and an example for information transmitted from the communication unit in the first device of FIG. 33.

FIG. 37 is a table showing a format and examples for data in the operation history storage unit in the first device of FIG. 33.

FIG. 38 is a flow chart for the operation of the retrieval unit in the first device of FIG. 33 in a case of document production.

FIG. 39 is a table showing a format and examples for data in the position operation information storage unit in the second device of FIG. 34.

FIG. 40 is a flow chart for the operation of the retrieval unit in the second device of FIG. 34.

FIG. 41 is a table showing a format and examples for data in the location name storage unit in the second device of FIG. 34.

FIG. 42 is an illustration of an exemplary display by the retrieval result display unit in the second device of FIG. 34.

FIG. 43 is a schematic block diagram for a functional configuration of a first device in the fourth specific embodiment of the system of FIG. 1.

FIG. 44 is a schematic block diagram for a functional configuration of a second device in the fourth specific embodiment of the system of FIG. 1.

FIG. 45 is a flow chart for the operation of the retrieval unit in the first device of FIG. 43 in a case of document look up or correction.

FIG. 46 is a table showing a format and examples for data in the operation history storage unit in the first device of FIG. 43.

FIG. 47 is a flow chart for the operation of the retrieval unit in the first device of FIG. 43 in a case of document production.

FIG. 48 is a schematic block diagram for a functional configuration of a second device in the fifth specific embodiment of the system of FIG. 1.

FIG. 49 is a flow chart for the operation of the retrieval unit in the sixth specific embodiment of the system of FIG. 1, in a case corresponding to the first and second specific embodiments.

FIG. 50 is a flow chart for the operation of the retrieval unit in the sixth specific embodiment of the system of FIG. 1, in a case corresponding to the fourth and fifth specific embodiments.

FIG. 51 is a flow chart for the operation of the retrieval result display unit in the seventh specific embodiment of the system of FIG. 1.

FIGS. 52A and 52B are illustrations of exemplary displays by the retrieval result display unit in the seventh specific embodiment of the system of FIG. 1 at the step 5101 in FIG. 51.

FIG. 53 is an illustration of an exemplary display by the retrieval result display unit in the seventh specific embodiment of the system of FIG. 1 after the step 5103 in FIG. 51.

FIGS. 54A and 54B are illustrations of exemplary displays by the retrieval result display unit in the seventh specific embodiment of the system of FIG. 1 in a case corresponding to the third or fourth specific embodiments.

FIG. 55 is a table showing a format and examples for data in the map database in the seventh specific embodiment of the system of FIG. 1.

FIG. 56 is a schematic block diagram for a functional configuration of the eighth specific embodiment of the system of FIG. 1.

FIG. 57 is an illustration of one exemplary document in the eighth specific embodiment of the system of FIG. 1.

FIG. 58 is a table showing a format and an example for data in the document data storage unit in the configuration of FIG. 56 for an exemplary document of FIG. 57.

FIG. 59 is a table showing a format and an example for data in the position database in the configuration of FIG. 56 for an exemplary document of FIG. 57.

FIGS. 60A and 60B are illustrations of other exemplary documents in the eighth specific embodiment of the system of FIG. 1.

FIG. 61 is a table showing a format and an example for data in the document data storage unit in the configuration of FIG. 56 for exemplary documents of FIGS. 60A and 60B.

FIG. 62 is a table showing a format and an example for data in the position database in the configuration of FIG. 56 for exemplary documents of FIGS. 60A and 60B.

FIG. 63 is an illustration of an exemplary document list display in the eighth specific embodiment of the system of FIG. 1.

FIG. 64 is an illustration of another exemplary document list display in the eighth specific embodiment of the system of FIG. 1.

FIG. 65 is a schematic block diagram for a functional configuration of the ninth specific embodiment of the system of FIG. 1.

FIGS. 66A and 66B are illustrations of exemplary schedule tables used in the ninth specific embodiment of the system of FIG. 1.

FIGS. 67A and 67B are illustrations of other exemplary schedule tables used in the ninth specific embodiment of the system of FIG. 1.

FIG. 68 is an illustration of an exemplary schedule display in the ninth specific embodiment of the system of FIG. 1.

FIG. 69 is a schematic block diagram for a functional configuration of the tenth specific embodiment of the system of FIG. 1.

FIGS. 70A and 70B are illustrations of exemplary documents in the tenth specific embodiment of the system of FIG. 1.

FIG. 71 is an illustration of an exemplary layout of the document in the tenth specific embodiment of the system of FIG. 1.

FIG. 72 is a table summarizing the column template and constraints used in the tenth specific embodiment of the system of FIG. 1.

FIG. 73 is an illustration of an exemplary arrangement type list display in the tenth specific embodiment of the system of FIG. 1.

FIG. 74 is an illustration of data stored in the layout information memory unit in the configuration of FIG. 69.

FIGS. 75A and 75B are illustrations of exemplary layout information displays used in the tenth specific embodiment of the system of FIG. 1.

FIGS. 76A, 76B and 76C are illustrations of exemplary interface displays used in the tenth specific embodiment of the system of FIG. 1.

FIGS. 77A, 77B and 77C are illustrations of other exemplary interface displays used in the tenth specific embodiment of the system of FIG. 1.

FIG. 78 is an illustration of still another exemplary interface display used in the tenth specific embodiment of the system of FIG. 1.

FIG. 79 is a schematic block diagram for a configuration for extracting scene change in the eleventh specific embodiment of the system of FIG. 1.

FIG. 80 is a flow chart for the operation of the histogram calculation unit in the configuration of FIG. 79.

FIG. 81 is a schematic block diagram for another configuration for extracting scene change in the eleventh specific embodiment of the system of FIG. 1.

FIG. 82 is a flow chart for the operation of the dispersion calculation unit in the configuration of FIG. 81.

FIG. 83 is a schematic block diagram for additional configuration for detecting the stopping of the scene change in the eleventh specific embodiment of the system of FIG. 1.

FIGS. 84A and 84B are schematic block diagrams for other configurations for extracting scene change in the eleventh specific embodiment of the system of FIG. 1.

FIG. 85 is a schematic block diagram for another configuration for extracting scene change in the eleventh specific embodiment of the system of FIG. 1.

FIG. 86 is a schematic block diagram for another configuration for extracting scene change in the eleventh specific embodiment of the system of FIG. 1.

FIG. 87 is a schematic block diagram for additional configurations for storing raw data in the eleventh specific embodiment of the system of FIG. 1.

FIGS. 88A and 88B are diagrams showing another scheme for the scene change detection in the eleventh specific embodiment of the system of FIG. 1.

FIG. 89 is an illustration of an exemplary display used in the human being detection in the eleventh specific embodiment of the system of FIG. 1.

FIG. 90 is an illustration of another exemplary display used in the human being detection in the eleventh specific embodiment of the system of FIG. 1.

FIGS. 91A and 91B are illustrations of one scheme for utilizing the images in the eleventh specific embodiment of the system of FIG. 1.

FIG. 92 is a diagram showing a scheme for realizing the scheme of FIG. 91 in the eleventh specific embodiment of the system of FIG. 1.

FIG. 93 is a diagram showing another scheme for realizing the scheme of FIG. 91 in the eleventh specific embodiment of the system of FIG. 1.

FIGS. 94A and 94B are diagrams showing another scheme for the retrieval in the eleventh specific embodiment of the system of FIG. 1.

FIG. 95 is an illustration of an exemplary display for the retrieval in the eleventh specific embodiment of the system of FIG. 1.

FIG. 96 is a schematic block diagram for a functional configuration of the twelfth specific embodiment of the system of FIG. 1.

FIG. 97 is a block diagram for a detailed configuration of the retrieval key analysis unit in the configuration of FIG. 96.

FIG. 98 is a table summarizing the retrieval key analysis result obtained in the configuration of FIG. 97.

FIG. 99 is an illustration of exemplary content of the retrieval word dictionary in the configuration of FIG. 97.

FIG. 100 is an illustration of exemplary content of the retrieval word extraction rule dictionary in the configuration of FIG. 97.

FIG. 101 is a table summarizing the operation events detected by the access information detection unit in the configuration of FIG. 96.

FIG. 102 is a table summarizing the attributes of access information elements used in the twelfth specific embodiment of the system of FIG. 1.

FIG. 103 is a table summarizing the meanings of variables used in the twelfth specific embodiment of the system of FIG. 1.

FIG. 104 is an illustration of an exemplary access information display used in the twelfth specific embodiment of the system of FIG. 1.

FIG. 105 is a flow chart for one operation of the access information detection unit in the configuration of FIG. 96.

FIG. 106 is a flow chart for another operation of the access information detection unit in the configuration of FIG. 96.

FIG. 107 is a flow chart for another operation of the access information detection unit in the configuration of FIG. 96.

FIG. 108 is a flow chart for another operation of the access information detection unit in the configuration of FIG. 96.

FIG. 109 is a flow chart for another operation of the access information detection unit in the configuration of FIG. 96.

FIG. 110 is a table summarizing keywords used in the twelfth specific embodiment of the system of FIG. 1.

FIG. 111 is a flow chart for one operation of the retrieval unit in the configuration of FIG. 96.

FIG. 112 is a flow chart for another operation of the retrieval unit in the configuration of FIG. 96.

FIG. 113 is a flow chart for another operation of the retrieval unit in the configuration of FIG. 96.

FIG. 114 is a flow chart for another operation of the retrieval unit in the configuration of FIG. 96.

FIG. 115 is a flow chart for another operation of the retrieval unit in the configuration of FIG. 96.

FIG. 116 is a flow chart for another operation of the retrieval unit in the configuration of FIG. 96.

FIG. 117 is an illustration of an exemplary display used in the twelfth specific embodiment of the system of FIG. 1.

FIG. 118 is an illustration of another exemplary display used in the twelfth specific embodiment of the system of FIG. 1.

FIG. 119 is an illustration of another exemplary display used in the twelfth specific embodiment of the system of FIG. 1.

FIG. 120 is an illustration of another exemplary display used in the twelfth specific embodiment of the system of FIG. 1.

FIG. 121 is an illustration of another exemplary access information display used in the twelfth specific embodiment of the system of FIG. 1.

FIG. 122 is an illustration of an exemplary document display used in the twelfth specific embodiment of the system of FIG. 1.

FIG. 123 is a schematic block diagram for a functional configuration of the thirteenth specific embodiment of the system of FIG. 1.

FIG. 124 is an illustration of an exemplary environmental information display used in the thirteenth specific embodiment of the system of FIG. 1.

FIG. 125 is a schematic block diagram for a functional configuration of the fourteenth specific embodiment of the system of FIG. 1.

FIG. 126 is a table showing a format and examples for data in the computer ID memory unit in the configuration of FIG. 125.

FIG. 127 is a table showing an example for data in the retrieval target data memory unit in the configuration of FIG. 125.

FIG. 128 is a flow chart for the operation of the computer ID setting unit in the configuration of FIG. 125.

FIG. 129 is a flow chart for one operation of the retrieval unit in the configuration of FIG. 125.

FIG. 130 is an illustration of a menu display used in the operation of FIG. 129.

FIG. 131 is an illustration of a retrieval result display used in the operation of FIG. 129.

FIG. 132 is an illustration of a correspondence table used in the fourteenth specific embodiment of the system of FIG. 1.

FIG. 133 is a flow chart for another operation of the retrieval unit in the configuration of FIG. 125.

FIG. 134 is an illustration of one display used in the operation of FIG. 133.

FIG. 135 is an illustration of another display used in the operation of FIG. 133.

FIG. 136 is a table showing a format and examples of the environmental information used in the fourteenth specific embodiment of the system of FIG. 1.

FIG. 137 is a flow chart for the operation of the environmental information output unit in the configuration of FIG. 125.

FIG. 138 is an illustration of an exemplary environmental information display used in the fourteenth specific embodiment of the system of FIG. 1.

FIGS. 139A and 139B are illustrations of another exemplary environmental information displays used in the fourteenth specific embodiment of the system of FIG. 1.

FIG. 140 is an illustration of still another exemplary environmental information display used in the fourteenth specific embodiment of the system of FIG. 1.

FIG. 141 is a schematic block diagram for a functional configuration of the second general embodiment of the data retrieval system according to the present invention.

FIG. 142 is a partially detailed block diagram for the configuration of FIG. 141 for a case of extracting background portion.

FIG. 143 is a partially detailed block diagram for the configuration of FIG. 141 for a case of extracting character portion.

FIG. 144 is a partially detailed block diagram for the configuration of FIG. 141 for a case of extracting specific portion.

FIG. 145 is a table showing an example of data stored in the data storage unit in the configuration of FIG. 141.

FIGS. 146A and 146B are block diagrams for detailed configurations of the retrieval unit in the configuration of FIG. 141.

FIG. 147 is a schematic block diagram for a functional configuration of the third general embodiment of the data retrieval system according to the present invention.

FIG. 148 is a flow chart for the time series data input operation in the system of FIG. 147.

FIG. 149 is a flow chart for the retrieval data output operation in the system of FIG. 147.

FIGS. 150A and 150B are a diagram and a table showing supplemented time series data used in the system of FIG. 147.

FIGS. 151A and 151B are illustrations of exemplary data formats in the time series data storage unit in the system of FIG. 147.

FIG. 152 is an illustration of an exemplary structural information used in the system of FIG. 147.

FIGS. 153A and 153B are illustrations of exemplary data formats in the structural information storage unit in the system of FIG. 147.

FIG. 154 is a block diagram for a detailed configuration of the structural analysis unit in the system of FIG. 147.

FIG. 155 is a graph of speech power analyzed in the configuration of FIG. 154.

FIG. 156 is an illustration of one exemplary structural information obtained by the configuration of FIG. 154.

FIGS. 157A and 157B are illustrations of another exemplary structural information obtained by the configuration of FIG. 154.

FIG. 158 is an illustration of another exemplary structural information obtained by the configuration of FIG. 154.

FIGS. 159A, 159B, 159C, 159D, and 159E are graphs of exemplary data and its analysis result in the system of FIG. 147.

FIG. 160 is a schematic block diagram of a configuration for estimating the location in the system of FIG. 147.

FIG. 161 is a schematic block diagram of a modified configuration for the third general embodiment of the data retrieval system according to the present invention.

FIGS. 162A and 162B are illustrations of exemplary displays used in the system of FIG. 161.

FIG. 163 is a schematic block diagram of a speech dialogue evaluation system utilizing the system of FIG. 147.

FIGS. 164A, 164B and 164C are illustrations of an exemplary output of the speech dialogue system in the configuration of FIG. 163.

FIG. 165 is an illustration of an exemplary window interface used in the system of FIG. 163.

FIGS. 166A and 166B are illustrations of examples of the structural information obtained in the system of FIG. 163.

FIG. 167 is an illustration of an exemplary output of the dialogue evaluation system in the configuration of FIG. 163.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

First, the basic concepts of the data retrieval system according to the present invention and the definitions of the terms used in the following description of the preferred embodiments will be described.

As represented by a word processor, in many of the present day information related system, the data entered into the system are often re-utilized after the manipulation. However, as a number of data entered into the system increases, there arises a problem that the retrieval of the data to be re-utilized requires a longer time.

In the data retrieval system according to the present invention, at a time of the system use, i.e., at a time of entering the data into the system or at a time of making an access to the data for the purposes of manipulation or editing, the environmental information indicating the situations of the user himself, the system itself, or their surrounding is utilized as the retrieval key for the data to be retrieved, where this environmental information is basically extracted automatically.

Conventionally, in a case of retrieving the necessary data out of many data already entered into the system, when the label consciously assigned to the data by the user such as the data file name is not available or inappropriate, the only available additional retrieval key is a physical label such as a data generation time which can be automatically assigned by the system relatively easily, and in many cases, the judgement of the correspondence between such a physical label and the data to be retrieved has largely relied on the memory of the user.

However, in a usual circumstance in which the data retrieval is required, it is more natural for the human user to relate the data to be retrieved with the situations of the user himself, the system itself, or their surrounding at a time the previous access to that data to be retrieved, such as "around a time at which this and that had happened", "when I was doing this and that", "when I was at so and so" "when I was with so and so", etc., rather than the physical label such as the time, so that the retrieval based on the above described physical label has often been quite inconvenient.

In the data retrieval system according to the present invention, the physical information called environmental analysis target data, which are obtainable from the situations of the user himself, the system itself, or their surrounding at a time of the previous access to the data to be retrieved, are automatically obtained and analyzed to produce the environmental information which is much more convenient to use as the retrieval key, and such an environmental information is stored in relation to the data to be retrieved and provided as the retrieval key whenever the retrieval is required.

Now, the first general embodiment of the data retrieval system according to the present invention will be described in detail.

In this first general embodiment, the system has a basic overall configuration as shown in FIG. 1. Namely, this data retrieval system comprises: a retrieval target data input unit 1 to which the retrieval target data is entered; a retrieval target data memory unit 2 which receives the output of the retrieval target data input unit 1; a retrieval target data output unit 3 which receives the output of the retrieval target data memory unit 2; an environmental analysis target data input unit 4 to which the environmental analysis target data is entered; an environmental information analysis unit 5 which receives the outputs of the retrieval target data input unit 1 and the environmental analysis target data input unit 4; a retrieval key input unit 8 to which a retrieval key is entered; a retrieval unit 9 which receives the output of the retrieval key input unit 8; an environmental information memory unit 6 which receives the outputs of the environmental information analysis unit 5 and the retrieval unit 9; and an environmental information output unit 7 which receives the output of the environmental information memory unit 6.

Here, as summarized in the table shown in FIG. 2, the retrieval target data are the data entered into the system in general. For example, the retrieval target data include the document or data file containing texts and images entered by utilizing the keyboard, mouse, pen, etc., the image data such as still pictures and dynamic pictures entered by utilizing the image scanner or camera, and the acoustic data recorded by the microphone or synthesized electronically, which are entered into the system, stored therein, and made available for the re-utilization by the operations such as editing, manipulation, and looking up.

As shown in FIG. 3, the retrieval target data input unit 1 further comprises input devices 1A for entering the retrieval target data such as a microphone, a camera, etc., and a data conversion unit 1B for attaching information related to the input data which will be required by the later analysis, such as a device identifier in a case of using more than one input devices 1A, a quantization bit number, a sampling frequency, etc., for example.

The retrieval target data memory unit 2 stores the retrieval target data entered by the retrieval target data input unit 1.

The retrieval target data output unit 3 takes out the retrieval target data stored in the retrieval target data memory unit 2 and outputs and presents them to the user. Here, the manner of output and presentation depends on the application. In a case of dealing with the document or image data, it is appropriate to provide means for displaying and printing, while in a case of dealing with the acoustic data, it is appropriate to provide means for realizing the acoustic reproduction and output such as a loudspeaker.

Also, as summarized in the table shown in FIG. 2, the environmental analysis target data are the source data from which the environmental information is extracted by the analysis, and which are entered into the system automatically at a time of the system use by the user.

For example, the environmental analysis target data can include the history of the input operation utilizing the keyboard, mouse, pen, etc., the image data such as the still pictures and the dynamic pictures entered by utilizing the scanner, camera, etc., the acoustic data recorded by the microphone or synthesized electronically, the position measurement data obtained by utilizing the position measurement device such as GPS (Global Positioning System) and magnetic sensor, and the data indicative of the external situation and the user's behavior at a time of the system use such as the measurement data obtained by the temperature, humidity, and pressure sensors, and the user's health condition measurement data such as the pulse, blood pressure, body temperature, blood sugar level, etc.

In addition, the environmental analysis target data may also include the input entered by the user at a time of the system use without an intention of being used for the retrieval. For example, as the workstation is usually used by a plurality of users, there is a need to enter the user name at a beginning of the system use, and this reveals the user ID according to which who has been using the system can be determined. Also, as the identifier of the computer connected through the network is acquired, it is possible to determine which system has been used. In a case the location of the system is fixed, it is also possible to determine where the system has been used. Similarly, who entered the data where can be determined by analyzing the input data other than the retrieval target data such as the schedule and the other documents. It is also possible to utilize the retrieval target data as the environmental analysis target data.

As shown in FIG. 4, the environmental analysis target data input unit 4 further comprises input devices 4A for entering the environmental analysis target data such as a pen, mouse, etc., and a data conversion unit 4B for attaching information related to the input data which will be required by the later analysis.

The environmental analysis target data is associated with the correspondence information indicating to which retrieval target data this environmental analysis target data corresponds. For example, when the retrieval target data is the time series data, the input time of the retrieval target data can be utilized for this correspondence information. Namely, by attaching an input time to each of the retrieval target data and the environmental analysis target data, it becomes possible to make a correspondence between them according to the attached time information.

Also, in a case the retrieval target data is either invariable or varying very slightly in time, or else the environmental analysis target data is invariable or varying very slightly in time, an identifier can be attached to each data such that it can be utilized as the correspondence information. Namely, by attaching a data identifier to each of the retrieval target data and the environmental analysis target data, it becomes possible to identify the corresponding data according to the data identifiers.

From this environmental analysis target data, the environmental information can be obtained by the analysis such as the extraction of the utterance content by the speech recognition, the distinction of the system use location by the acoustic characteristic, the identification of the speaker by the speaker recognition, the identification of the system use location by the image recognition, the layout analysis of the document data, and the analysis of the schedule or the data access information, for example.

Such an analysis is carried out at the environmental information analysis unit 5. Here, the actual analysis to be carried out depends on the form of the environmental analysis target data, and the higher level environmental information may be obtained by analyzing a plurality of analysis results and structuring them. The concrete details of the analysis of the environmental analysis target data and the resulting environmental information will be described in conjunction with the specific embodiments below.

It is preferable for this analysis to be carried out automatically at a time of the system use, but in a case the analysis processing is complicated and the real time processing cannot be guaranteed, the analysis may be carried out while the user is not using the system.

Also, in a case the user wishes to correct the errors in the automatically obtained analysis result or add the environmental information intentionally, it is possible to enable the user to make the manual correction and addition on the analysis result.

Similarly to the environmental analysis target data which is associated with the correspondence information as described above, the environmental information is also associated with the correspondence information indicating from which environmental analysis target data this environmental information has been obtained by the analysis. Here, this correspondence information may be that which makes the correspondence between the retrieval target data and the environmental information instead. Just as the correspondence information between the retrieval target data and the environmental analysis target data, this correspondence information associated with the environmental information can be given by the input time of the retrieval data in a case the data change in time is significant, or by the data identifier in a case the data change in time is insignificant.

Also, in a case the obtained environmental information corresponds to a part of the retrieval target data or the environmental analysis target data, by attaching the correspondence information indicating which part of the whole it is corresponding to, it becomes possible to carry out the retrieval more easily and accurately.

By tracing the correspondence information among the environmental information, the environmental analysis target data, and the retrieval target data, it becomes possible to carry out the retrieval of the retrieval target data by using the environmental information as the retrieval key.

In addition, the environmental information may also be attached with the information indicating by what kind of analysis it has been obtained, or under what likelihood it has been obtained. In such a case, such an additional information can be utilized for judging which environmental information among those obtained by a plurality of analysis methods is most reliable.

Here, as summarized in the table shown in FIG. 2, the environmental information is the information related to the user himself or his surrounding situation (time, location), at a time the user inputs, edits, retrieves, or outputs the retrieval target data by using the system, which is not just simple physical time and location, but the structured information meaningful to the user in carrying out the retrieval, such as "location at which the user was present at a certain time", "time in which the user was at a certain location", "location and time at which the user was doing a certain thing", "location and time at which the user was in a certain state", "location and time at which the user was with a certain person", etc. For example, the environmental information is related to a time at which the user was with somebody, a time at which the user attended the conference, a time at which the user was in a train to the company, a location of the living room of the user's home, etc. By linking such an environmental information with the retrieval target data, it becomes possible to search out the retrieval target data required by the user easily, using the environmental information as the retrieval key.

Each environmental information can be made easier to recognize for the user in a case of presenting the content of the environmental information to the user, by attaching the identifier in accordance with the topic obtained by the analysis to the concrete analysis result. For example, such an identifier to be attached may include "who entered the retrieval target data", "where the retrieval target data had entered", "how the retrieval target data had entered", "why the retrieval target data had entered", etc.

Such an environmental information is obtained by analyzing the physical data such as the sounds automatically taken while the user uses the system, the signals such as the video signals, and the measurement values for position, etc. Here, the data which are the target of the analysis for obtaining the environmental information are referred as the environmental analysis target data as summarized in the table shown in FIG. 2.

The environmental information analysis unit 5 analyzes and structures the environmental analysis target data entered into the system through the environmental analysis target data input unit 4, to generate the environmental information.

The environmental information memory unit 6 stores the environmental information containing the analysis result as well as its structural information, obtained by the environmental information analysis unit 5.

The environmental information output unit 7 outputs the environmental information containing the analysis result as well as its structural information, stored in the environmental information memory unit 6.

The retrieval key input unit 8 includes an input device such as a keyboard, a touch panel, a pen, and a mouse for admitting the input from the user, and an environmental information conversion unit for converting the input from the user into the desired environmental information to be retrieved.

The retrieval unit 9 retrieves the environmental information specified by the user at the retrieval key input unit 8 from the environmental information memory unit 6, and takes out the retrieval target data linked with that environmental information.

In addition, the retrieval unit 9 is also capable of tracing the environmental information and the structural information stored in the environmental information memory unit 6 to obtain the related environmental information.

In contrast to the above described data retrieval system according to the present invention, the conventional word processor, electronic publishing system, or input/output application for multi-media data such as sound and video recording system is only realizing the function to control the input/output and storing of the retrieval target data, as it is only equipped with the equivalents of the retrieval target data input unit 1, the retrieval target data memory unit 2, and the retrieval target data output unit 3 in the configuration of FIG. 1.

On the other hand, the present invention provides the framework for searching out the retrieval target data by utilizing the environmental information indicating the situation of the user and the surrounding at a time the user was manipulating the retrieval target data on the system, by linking the environmental information with the retrieval target data. In particular, it is characterized in that the generation of the environmental information is made automatically by the system in principle.

Now, further details of the individual operations involved in this data retrieval system according to the present invention will be described.

<Input of data from the input device>

The input device 1A or 4A of the retrieval target data input unit 1 or the environmental analysis target data input unit 4 enters the continuous data such as the acoustic media or video media, the relative position data of the input device and the system indicating the pointing position, and the absolute position data of the system itself obtained by utilizing the position measurement system, etc. into the system.

The data conversion unit 1B or 4B of the retrieval target data input unit 1 or the environmental analysis target data input unit 4 converts the input data into a format suitable for handling within the system. For example, the continuous data are stored and processed in a discretized and quantized format. Here, the discretized and quantized retrieval target data and environmental analysis target data are attached with the discretization and quantization information as well. The discretization and quantization information will be utilized at a time of reproducing and outputting a part or a whole of the retrieval target data. In the following, the retrieval target data and the environmental analysis target data referred within the context of this data retrieval system are assumed to be the converted ones.

<Handling of time information as the environmental analysis target data>

When the operations with respect to the retrieval target data such as input, editing, retrieval, and output are regarded as events, the occurrence time of each event can be utilized as the analysis target for obtaining the environmental information related to the retrieval target data. Namely, the event occurrence time series such as the time information indicating the times at which the retrieval target data are entered into the system can be handled as the environmental analysis target data. This time information can provide the records as to when the retrieval target data has been operated, or when each part of the retrieval target data has been operated.

Similarly, for the environmental analysis target data, the time information indicating its input time can be handled as the environmental analysis target data.

By providing the time information for each input data in this manner, it becomes possible to make the timewise correspondences among the data. Thus, the time information can be utilized in establishing the timewise link between the retrieval data or its part and the environmental analysis target data, or the timewise link between the operation event with respect to the retrieval target data and the environmental analysis target data.

<Storing of the retrieval target data and the environmental analysis target data>

The retrieval target data memory unit 2 and the environmental analysis target data memory unit 6 store the retrieval target data and the environmental analysis target data, respectively, in association with the additional information such as the discretization and quantization information.

Here, the main body of the retrieval target data or the environmental analysis target data may be linked with the additional information, if desired. Alternatively, in view of the fact that the main body of the retrieval target data or the environmental analysis target data usually requires a large memory capacity while the additional information is often commonly shared among the input devices or input data, each of the retrieval target data or the environmental analysis target data may be provided with a frame having a pointer for pointing the main body of the data and a pointer for pointing the additional information or the main body of the additional information.

<Timing for environmental analysis at the environmental information analysis unit 5>

The environmental information analysis unit 5 obtains the environmental information by analyzing the environmental analysis target data, using the processing such as the parameter extraction and the pattern recognition and comprehension.

Here, as indicated in FIG. 5, the analysis of the environmental analysis target data is carried out automatically when the environmental analysis target data are entered. As already mentioned, the environmental analysis target data are entered whenever the user has been manipulating the retrieval target data on the system, so that the analysis is going to be carried out throughout the period in which the user is using the system.

<Analysis of the environmental information at the environmental information analysis unit 5>

The information obtained by the environmental information analysis unit 5 through the extraction or the recognition and comprehension processing is the information indicating the situation of the user himself and his surrounding at a time the user is using the system either directly or indirectly.

As an example, a case of using the fact that the retrieval target data had been produced at the living room of the user's home as the environmental information will be described here. This environmental information can be obtained from the position data series transmitted from the position measurement system such as the GPS as the environmental analysis target data, by recognizing the fact that its location is the living room of the user's home. The environmental information indicating the use location can also be obtained from the environmental analysis target data other than that obtained by the GPS, such as the reverberation data, the surrounding sound data, the image data, and the behavior history data, by the appropriate processing, as indicated in FIG. 6.

Now, it is necessary for the environmental information analysis unit 5 to have the automatic interpretations set up in advance as to which environmental information is to be obtained, from which analysis result can this environmental information be obtained, and from which environmental analysis target data and by which analysis method can this analysis result be obtained. In the above example, for the environmental information concerning the system use location, before the analysis, it is necessary to set up the facts including the use location to be recognized such as his own working place, the use of the GPS position measurement data as the environmental analysis target data to be used in recognizing that use location, the correspondence between the position measurement data value and the use location to be recognized, etc.

Among these automatic interpretations, "which environmental analysis target data should be analyzed in which analysis method in order to obtain a certain environmental information" is hardly dependent on the situations or the users, so that this information can be provided in the system as the default knowledge concerning the analysis in advance. In the examples shown in FIG. 6, in order to obtain the environmental information indicating the use location, it suffices to analyze the reverberation pattern of the reverberation data, the surrounding sound pattern of the surrounding sound data, etc., and this information can be provided as the preliminary knowledge provided in advance in the system regardless of the applications.

On the other hand, "which (instance of) environmental information is to be obtained, or from which analysis result can this (instance of) environmental information be obtained" is largely dependent on the situations and the users. Namely, the former is the knowledge concerning what is to be recognized, such as the living room of the own home, the working place, and the car for transportation which can be three locations to be recognized as the environmental information indicating the use location. On the other hand, the latter is the knowledge concerning the comprehension of the analysis result, such as it should be recognized as the living room of the own home when what kind of the reverberation pattern or what kind of surrounding sound pattern is obtained as a result of the analysis. Both of these are knowledges which are largely dependent on the requests from the applications or the users. Thus, it is necessary for the user to set up the portions where the individual environmental information is obtained from the analysis result, before the analysis takes place. The concrete examples include the sound, image, position, schedule, access information, etc. which will be described in detail below.

<Mechanism of retrieval>

Each analysis result is attached with the information concerning the analysis such as it is obtained by the analysis of which part of the environmental analysis target data. By this information, the individual environmental information is linked with a part or a whole of the environmental analysis target data that had been analyzed. As already mentioned, there is a link between the retrieval target data and the environmental analysis target data, so that it becomes possible to retrieve the retrieval target data from the environmental information.

In the retrieval operation, the environmental information linked with the retrieval target data is searched out first. For example, the environmental informations indicating the use location, the use time, the use state, etc. are displayed through the environmental information output unit 7, and the user selects the appropriate environmental information to be used as the retrieval key from the displayed environmental informations. Here, the environmental information to be used as the retrieval key is specified through the retrieval key input unit 8.

The retrieval unit 9 then searches out the specified environmental information in the environmental information memory unit 6, and the retrieval target data linked with the searched out environmental information is outputted from the retrieval target data output unit 3 as the retrieval result.

<Attachment of the likelihood to the environmental information at the environmental information analysis unit 5>

The possibility for being accurate (likelihood) of the analysis result depends on the analysis method. In addition, even among the results obtained by the same analysis method, the likelihood varies from one analysis result to another.

Consequently, as the information concerning the analysis described above, the information indicating which analysis method it is obtained from and what level of the likelihood it has according to that analysis method is attached. This likelihood may be the value subjectively determined by the user, or the value set up by the system semi-automatically.

For example, for the higher frequency to use the environmental information obtained by a particular analysis method as the retrieval key, the likelihood of that particular analysis method can be set to be higher. Also, when the analysis is some kind of the pattern recognition, the likelihood can be given by the similarity to the reference pattern.

When there are more than one analysis methods and the analysis results obtained by these analysis methods contradicts with each other, by looking up the information concerning the likelihood of the analysis method and the analysis result and giving a higher priority to those having the higher likelihood, it becomes possible to carry out the retrieval of the retrieval target data more efficiently. It is also possible to give a higher priority to the environmental information which has been obtained by many analysis methods with the high likelihoods.

<Structuring of the analysis result at the environmental information analysis unit 5>

In a case the multi-media data are available as the environmental analysis target data, the analysis is carried out for each medium separately first.

Then, the analysis result for each medium is structured for each environmental information to be obtained. For example, the environmental information concerning the use location indicating where the retrieval target data had been entered can be obtained by various methods, such as the position measurement system, the recognition by the acoustic data such as the reverberation characteristic or the surrounding sound, the scene recognition using the images, the estimation from the data such as the schedule or the ordinary behavior history, as indicated in FIG. 6. The analysis results concerning the use location obtained by these methods using a plurality of media are structured as the environmental informations with the same semantic content.

<Use of the retrieval target data as the environmental analysis target data >

In the above, the retrieval target data and the environmental analysis target data are described as if they are clearly distinct data, but these data may be identical. Namely, the retrieval target data can be directly used as the environmental analysis target data to obtain the environmental information by the analysis. For example, in a case of reproducing the utterance of a particular person by using the recognition result obtained by the speaker recognition applied to the speech data entered from the microphone as the environmental information, the retrieval target data as well as the environmental analysis target data are given by the identical speech data.

In this manner, it is possible within the system for a certain data to be the retrieval target from the viewpoint of the retrieval and the analysis target from the viewpoint of the analysis of the environment, simultaneously.

<Handling of the device information as the environmental analysis target data>

The system can be equipped with a plurality of input devices in the retrieval target data input unit 1 or the environmental analysis target data input unit 4. In a case of using a plurality of input devices, the device information indicating from which input device each input data has been entered can be used as the environmental analysis target information. For example, the environmental information can be obtained from the information such as it is the data in a video media or acoustic media, it is the data entered by the input device located where, it is the data entered by the microphone or camera oriented toward which direction, etc.

<Attachment of the likelihood by the device information>

In a case the quality of the environmental analysis target data differs from one input device to another, it is also possible to change the likelihood according to the device information.

Next, the modification of the configuration of FIG. 1 to enable the input of the environmental information from the user will be described with reference to FIG. 7.

This configuration of FIG. 7 differs from that of FIG. 1 in the incorporation of the environmental information input unit 10 which includes an input device such as a keyboard, a touch panel, a pen, and a mouse for admitting the input from the user, and an environmental information conversion unit for converting the entered input into the environmental information.

In general, it is unavoidable for the result obtained by the automatic analysis to contain some errors, and the direct use of such a result containing errors, without making any correction, can cause the lowering of the retrieval efficiency. In this configuration of FIG. 7, by allowing the user to enter the environmental information directly, it is possible to correct or delete the analysis errors made by the system and to add the necessary environmental information.

Here, the environmental information that can be entered by the user contains the system analysis result, its structured information, and its likelihood. It is also possible to attach a higher likelihood to the environmental information directly entered by the user. Also, for the analysis method whose analysis results are frequently deleted by the user, the likelihood of that analysis method can be lowered. On the contrary, the analysis method which obtained many environmental informations which are repeatedly used as the retrieval keys with frequent corrections can be judged as that which yields the analysis results to which the user's attentions are paid, so that the likelihood of this analysis method can be raised.

Next, the modification of the configuration of FIG. 1 to enable the storing of the environmental analysis target data will be described with reference to FIG. 8.

This configuration of FIG. 8 differs from that of FIG. 1 in the incorporation of the environmental analysis target data memory unit 11 which stores the environmental analysis target data entered from the environmental analysis target data input unit 4. The environmental analysis target data stored in this environmental analysis target data memory unit 11 are read out according to the need, and supplied to the environmental information analysis unit 5 for the analysis and the structuring.

By analyzing the environmental analysis target data already entered into the system once, it becomes possible to re-analyze the environmental analysis target data in a case the analysis excessively complicated for the hardware performance level is to be carried out off line, or in a case the already obtained environmental information is not sufficient. Namely, in the analysis of the environmental analysis target data to obtain the environmental information, there is a need to decide what is to be taken out by the analysis (analysis result), and how it is to be analyzed (analysis method including the necessary parameter setting), but depending on the purpose of the retrieval, it is possible for the already obtained analysis result or the analysis method is insufficient. In this configuration of FIG. 8, the environmental analysis target data are stored within the system, so that the environmental information suitable for the need of the time can be obtained by resetting the analysis method or the analysis result to be obtained.

Now, various specific embodiments for the above described first general embodiment of the data retrieval system according to the present invention will be described in detail.

First, the first specific embodiment concerns with a case in which the data ID and the position information indicating the data production or retrieval location are stored in correspondence at a time of the data production or retrieval, and the desired data is retrieved by using this position information. For example, the data obtained by the GPS which is capable of collecting the user's behavior history can be used as the position information, and the extraction of the position information can be carried out in accordance with the information concerning the maps and the diagrams of rooms. In the following, the retrieval target data is assumed to be the document data for the sake of definiteness.

This first specific embodiment provides a document retrieval system in a form of a compact device having a physical configuration as shown in FIG. 9, which comprises: a CPU 101 for controlling an overall operation of the system; a memory unit 102 such as a semiconductor memory, a magnetic disk, an optical disk, a CD-ROM, an IC card, etc. which stores the programs and the data; a display unit 104 such as a liquid crystal display, a plasma display, etc. for displaying the retrieval result and the document content, under the control of a display controller 103; an input unit 106 such as a keyboard, a mouse, etc. for making the input of the retrieval command from the user, under the control of an input controller 105; and a communication unit 107 such as a GPS antenna, an infrared receiver, etc. for communicating with the external of the system, all of which are connected through a bus line.

The functional configuration of this first specific embodiment is as shown in FIG. 10, which comprises: a document data storage unit 25 for storing the document data; an input unit 20 for entering the retrieval command; a retrieval unit 23 for retrieving the document data according to the retrieval command; a retrieval result display unit 22 for displaying the retrieval result; a position measurement unit 24 for measuring a position at which the system itself is located; a position and document information storage unit 26 for storing the measured position information and the retrieved document name; a location name storage unit 27 for storing the measured position information and the location name representing the measured position; and a control unit 21 for controlling operation of the above mentioned elements.

In this case, the document data in the document data storage unit 25 is expressed in a format as shown in FIG. 11, which comprises a document name and a document ID of a file in which the document data is stored. In the exemplary data shown in FIG. 11, "file 8", "file 11", and "file 23" represent the document IDs. The document names are arranged in a prescribed order such as that of the JIS (Japanese Industrial Standards) codes. The document data storage unit 25 is physically realized on the memory unit 102 in FIG. 9, and can be exchanged as in a case of the CD-ROM or the IC card.

Now, the processing in a case of retrieving a specific document will be described in detail.

In this case, the user enters the name of the document to be retrieved through the input unit 20 such as a keyboard, and either specifies "retrieval" command on the display window (now shown) through the input unit 20 such as a mouse, or presses "retrieval" key provided on the keyboard. Here, the control unit 21 operates according to the flow chart of FIG. 12, while the retrieval unit 23 operates according to the flow chart of FIG. 13 as follows.

Namely, when the "retrieval" is selected, the control unit 21 receives the input character string from the input unit 20 (step 1201), activates the retrieval unit 23 (step 1202), and transfer the entered input character string to the activated retrieval unit 23 (step 1203). Then, the retrieval unit 23 retrieves the document data by using the input character string transferred from the control unit 21 (step 1301), and when a pertinent document name is found (step 1302 yes), the obtained document name and document ID are transferred to the control unit 21 (step 1303).

Next, the control unit 21 receives the retrieval result containing the document name and the document ID transferred from the retrieval unit 23 (step 1204), activates the retrieval result display unit 22 (step 1205), and transfer the received retrieval result to the activated retrieval result display unit 22 (step 1206). In response, the retrieval result display unit 22 generates the window for the retrieval result display and displays the content of the retrieved document data on that window. In this manner, the user can look up the desired document. It is also possible to enable the user to make a correction on the displayed document, in addition to the looking up.

Next, the retrieval unit 23 activates the position measurement unit 24 (step 1304). In response, the position measurement unit 24 calculates the latitude and longitude of a location at which the system is located, and transfers the position information containing the calculated latitude and longitude to the retrieval unit 23. Here, the method of position measurement to be used at the position measurement unit 24 is not essential to the present invention, and can be provided by any known position measurement method. In a case of using the GPS, it is possible to measure the position at the outside where the positioning satellite can be utilized.

Then, the retrieval unit 23 receives the position information transferred from the position measurement unit 24 (step 1305), and stores the received position information in correspondence with the obtained document ID of the retrieved document data in the position and document information storage unit 26 (step 1306). Here, the data format and the exemplary data in the position and document information storage unit 26 are as shown in FIG. 14. Here, in a case a plurality of document data are retrieved, instead of storing the document IDs of all the retrieved document data, it is possible to store only the document ID of the document data which is selected by the user on the retrieval result display unit 22 to display its content.

In addition to the looking up and the correction of the already existing document as described above, the user can also produce the new document. In this case, the input unit 20 stores the input entered by the user into the document data storage unit 25 along with the document ID and the document name specified by the user, and transfers the document ID to the control unit 21, and the control unit 21 activates the retrieval unit 23 and transfers the received document ID to the retrieval unit 23. In response, the retrieval unit 23 operates according to the flow chart of FIG. 15 as follows.

Namely, the retrieval unit 23 receives the document ID from the control unit 21 (step 1501), and activates the position measurement unit 24 (step 1502). Then, the retrieval unit 23 receives the position information from the position measurement unit 24 (step 1503), and stores the position information and the document ID in the position and document information storage unit 26 (step 1504).

Next, the processing in a case of setting up the desired location name in the system will be described in detail.

In this case, the user enters the name representing the desired location through the input unit 20, and either specify "location registration" command on the display window (now shown) through the input unit 20 such as a mouse, or presses "location registration" key provided on the keyboard. Here, the control unit 21 operates according to the flow chart of FIG. 16, while the position measurement unit 24 operates according to the flow chart of FIG. 17 as follows.

Namely, when "location registration" is selected, the control unit 21 receives the input character string from the input unit 20 (step 1601), activates the position measurement unit 24 (step 1602), and transfers the entered input character string to the activated position measurement unit 24 (step 1603). Then, the position measurement unit 24 receives the input character string transferred from the control unit 21 (step 1701), calculates the latitude and longitude of the position at which this computer is located (step 1702), and stores the the position information containing the calculated latitude and longitude in correspondence with the transferred input character string in the location name storage unit 27 (step 1703). Here, the data format and the exemplary data in the location name storage unit 27 are shown in FIG. 18.

Next, the processing in a case of retrieving the document which had been looked up, corrected, or produced according to the location name will be described.

In this case, at the desired timing and location, the user either specifies "location retrieval" command on the display window (now shown) through the input unit 20 such as a mouse, or presses "location retrieval" key provided on the keyboard. Here, the control unit 21 operates according to the flow chart of FIG. 19, while the retrieval result display unit 22 operates according to the flow chart of FIG. 20 as follows.

Namely, when the "location retrieval" is selected, the control unit 21 activates the retrieval result display unit 22 (step 1901) first. In response, the retrieval result display unit 22 takes out all the location names stored in the location name storage unit 27 (step 2001), displays these location names on the window for the retrieval result display (step 2002), and awaits the selection of one location name from the displayed location names by the user through the mouse (step 2003). Here, the exemplary display on the window for the retrieval result display appears as shown in FIG. 21, where the location name accompanied by the blacked box is the one selected by the user. The retrieval result display unit 22 then takes out the position information corresponding to the selected location name from the location name storage unit 27 (step 2004), and transfers that position information to the control unit 21 (step 2005).

Then, the control unit 21 receives the position information transferred from the retrieval result display unit 22 (step 1902), activates the retrieval unit 23 (step 1903), and transfer the received position information to the activated retrieval unit 23 (step 1904). Here, the retrieval unit 23 operates according to the flow chart of FIG. 22 as follows.

Namely, the retrieval unit 23 retrieves the document ID from the position and document information storage unit 26 by using the transferred position information (step 2201). Here, the retrieval processing proceeds according to the flow chart shown in FIG. 23. That is, the coincidences of the latitude and the longitude are judged within a prescribed range such as .+-.0.3.degree. (steps 2301, 2302), and retrieves the document ID of the document with the coinciding latitude and longitude values as the retrieval result (step 2303). Then, the retrieval unit 23 retrieves the document name from the document data storage unit 25 by using the retrieved document ID in the position and document information storage unit 26 (step 2203), and transfers the retrieved document name and document ID to the control unit 21 (step 2204).

The control unit 21 then receives the retrieval result containing the retrieved document name and document ID transferred from the retrieval unit 23 (step 1905), and transfers the received retrieval result to the retrieval result display unit 22 (step 1906).

The retrieval result display unit 22 then receives the retrieval result transferred from the control unit 21 (step 2006), and displays the received retrieval result on the window for the retrieval result display (step 2007). Here, in a case there is only one document in the retrieval result, the document name and the content of the document identified by the document ID are displayed, whereas when there are more than one documents in the retrieval result, a list of the document names is displayed as shown in FIG. 24. In the latter case, the user can select the desired document name from the displayed document names, and the retrieval result display unit 22 displays the content of the document identified by the document ID corresponding to the selected document name on the window for the retrieval result display.

It is also possible to modify the display of FIG. 21 such that, for each location name, all the relevant document names are also displayed simultaneously. In this case, it suffices to carry out the processing of FIG. 22 not just for the position information of the location selected by the user, but also for the position informations of all the location names stored in the location name storage unit 27.

Also, for the documents looked up, corrected, or produced by the user under the prescribed condition, or all the documents produced by the user, whether the location name corresponding to the position information of each document has been set up or not can be checked, and in a case there is a document for which the corresponding location name has not been set up, it is possible urge the user to register the location name. In this case, the control unit 21 operates according to the flow chart of FIG. 25 as follows.

Namely, when a number of documents looked up, corrected, or produced by the user exceeds a prescribed number (step 2501 yes), the control unit 21 activates the retrieval result display unit 22 (step 2502). Here, the retrieval result display unit 22 operates according to the flow chart of FIG. 26 as follows.

The retrieval result display unit 22 takes out all the position informations stored in the location name storage unit 27 (step 2601), and transfers these position informations to the control unit 21 (step 2602). The control unit 21 then receives the position informations transferred from the retrieval result display unit 22 (step 2503), activates the retrieval unit 23 (step 2504), and transfer the received position informations to the retrieval unit 23 (step 2505). Then, the retrieval unit 23 operates according to the flow chart of FIG. 27 as follows.

The retrieval unit 23 receives the position informations transferred from the control unit 21 (step 2701), and retrieves the document IDs from the position and document information storage unit 26 by using the received position informations (step 2702). When there is a position information among the position informations in the position and document information storage unit 26 which is not within a prescribed range from the transferred position informations, i.e., the position informations with the registered location names (step 2703 yes), the retrieval unit 23 retrieves the document name from the document data storage unit 25 by using the document ID corresponding to that position information (step 2704), and transfers the retrieved document name and document ID along with that position information to the control unit 21 (step 2705).

The control unit 21 then transfers the document name, document ID, and position information transferred from the retrieval unit 23 to the retrieval result display unit 22 (step 1706). The retrieval result display unit 22 then receives the document name, document ID, and position information transferred from the control unit 21 (step 2603), takes out the location name of the position information in the location name storage unit 27 which is closest to the received position information (step 2604), and displays the message for urging the user to register the location name, along with the received document name and the the location name taken out from the location name storage unit 27 as the closest one, on the window for the retrieval result display (step 2605).

Here, the exemplary display on the window for the retrieval result display appears as shown in FIG. 28. At this point, unless ›FINISH! command is selected by the user (step 2606 no), when the user enters the location name (step 2607 yes), the location name is retrieved from the location name storage unit 27 by using the entered location name (step 2608). Then, unless there is a coinciding location name (step 2609 no), the entered location name and the position information of the corresponding document are stored in the location name storage unit 27 (step 2610). Then, among the documents displayed, when there is a document with the position information which is out of the prescribed range such as .+-.0.5.degree. from the stored position information (step 2611 yes), the document name and the closest location name of such a document are displayed again (step 2612) and the operation returns to the step 2606. Here, the exemplary display after the registration of the location name by the user in response to the display of FIG. 28 appears as shown in FIG. 29.

In this first specific embodiment described above, the position measurement method is not limited to that using the GPS. For example, the infrared transmitter and receiver can be used as follows. Namely, the infrared transmitter which generates the characteristic bit pattern is provided at the desired location such as the conference rooms or working places in buildings or houses within a different sites or grounds, or in different buildings, houses, or floors in the same site or ground. Then, the infrared receiver can be used as the position measurement unit in the above embodiment, by receiving the bit pattern generated by the infrared transmitter instead of measuring the latitude and longitude, and then setting up the appropriate location name or room name.

Also, in this first specific embodiment described above, instead of selecting the displayed location name, the location name used for the retrieval may be entered. Namely, when the location name is entered, the entered location name is matched with the location names in the location name storage unit 27, and when there is a coinciding location name, the retrieval from the position and document information storage unit 26 is carried out by using the latitude and longitude values of that coinciding location name, so as to display the document with the document ID for which the latitude and longitude values are within the prescribed range.

Similarly, the latitude and longitude values for the retrieval may be entered. Namely, when the latitude and longitude values are entered, the retrieval from the position and document information storage unit 26 is carried out by using the entered latitude and longitude values, so as to display the document with the document ID for which the latitude and longitude values are within the prescribed range. In this case, the retrieval from the location name storage unit 27 by using the entered latitude and longitude values can be carried out at the same time, such that the location name for which the latitude and longitude values are within the prescribed range can be displayed simultaneously.

Also, in addition to the storing and displaying of the location name and the document name, the time can be measured at the same time, and a set of the location name, the document name, and the time can be stored. In this case, the document name can be displayed along with the time, for each location name.

Moreover, this first specific embodiment is not limited to the cases of looking up, correcting, and producing the document as described above. For example, by combining with the portable telephone, the position information concerning the use of the telephone can be stored, and it is possible to retrieve the information indicating where the telephone call was made with respect to where or who later on. Similarly, by combining with the electronic camera or video recorder, the position information concerning the picture or video taking location can be stored, and it is possible to retrieve the information indicating where the picture or video of what was taken later on. Also, by combining with the moving object or transportation means such as the automobile, the position information during the moving can be stored, and it is possible to retrieve the information indicating which route was taken later on.

Next, the second specific embodiment concerns with a modification of the first specific embodiment described above. In this second specific embodiment, the physical configuration of the system is the same as in FIG. 9 described above, while the functional configuration of the system is as shown in FIG. 30, which differs from that of FIG. 10 in that the retrieval unit 23 and the location name storage unit 27 are connected with each other such that, instead of providing the registration of the location name which is independent from the looking up, correcting, and producing of the documents as in the first specific embodiment, the presence or absence of the already registered location name is checked when the user looks up, corrects, or produces the document, and the registration is urged in a case of the absence.

In this second specific embodiment, in a case the user looks up or corrects the document, the retrieval unit 23 operates according to the flow chart of FIG. 31 as follows.

Namely, in the flow chart of FIG. 31, the steps 3101 to 3106 are identical to the steps 1301 to 1306 in FIG. 13 described above. After the step 3106, the location name is retrieved from the location name storage unit 27 by using the position information stored at the step 3106 to check if there is an already registered location name (step 3107), and if such an already registered location name is absent, i.e., when there is no position information which is within the prescribed range such as .+-.0.5.degree. from the stored position information (step 3108 no), the retrieval unit 23 notifies the control unit 21 that the location name corresponding to that position information is not set up, so as to urge the registration of the location name (step 3109).

Similarly, in a case the user produces the document, the retrieval unit 23 operates according to the flow chart of FIG. 32 as follows.

Namely, in the flow chart of FIG. 32, the steps 3201 to 3204 are identical to the steps 1501 to 1504 in FIG. 15 described above. After the step 3104, the location name is retrieved from the location name storage unit 27 by using the position information stored at the step 3204 to check if there is an already registered location name (step 3205), and if such an already registered location name is absent, i.e., when there is no position information which is within the prescribed range such as .+-.0.5.degree. from the stored position information (step 3206 no), the retrieval unit 23 notifies the control unit 21 that the location name corresponding to that position information is not set up, so as to urge the registration of the location name (step 3207).

Next, the third specific embodiment concerns with a modification of the first specific embodiment described above in which the data retrieval system comprises a first compact portable device having means for looking up, correcting, and producing the document and means for transmitting information, and a second device having means for receiving information and means for retrieving.

In this third specific embodiment, each of the first and second devices has the physical configuration similar to that of FIG. 9 described above, where the first device has the infrared transmitter as the communication unit 107 while the second device has the infrared receiver as the communication unit 107.

The first device has the functional configuration as shown in FIG. 33, which comprises: a document data storage unit 25 for storing the document data; an input unit 20 for entering the retrieval command; a retrieval unit 23 for retrieving the document data according to the retrieval command; a retrieval result display unit 22 for displaying the retrieval result; a timer unit 29 for measuring time; an operation history storage unit 31 for storing the time data and the document ID and document name of the document in the retrieval result; a communication unit 30 for notifying the presence of the first device; an output unit 28 for outputting the data stored in the operation history storage unit 31; and a control unit 21 for controlling operation of the above mentioned elements. This first device can be provided in plurality.

On the other hand, the second device has the functional configuration as shown in FIG. 34, which comprises: an input unit 20 for entering the retrieval command and the data input; a retrieval unit 23 for retrieving the document data according to the retrieval command; a retrieval result display unit 22 for displaying the retrieval result; a communication unit 30 (30-1 to 30-N corresponding to a plurality of the first devices) for receiving the information transmitted from the first device; a position operation information storage unit 32 for storing the information transmitted from the first device; a location name storage unit 27 for storing the location name representing the measured position; and a control unit 21 for controlling operation of the above mentioned elements.

Here, the document data in the document data storage unit 25 of the first device of FIG. 33 is expressed in the same format as that of FIG. 11 described above, while the control unit 21 in the first device of FIG. 33 operates according to the same flow chart as that of FIG. 12 described above.

The retrieval unit 23 in the first device of FIG. 33 operates according to the flow chart of FIG. 35 as follows. Namely, the retrieval unit 23 retrieves the document data by using the input character string transferred from the control unit 21 (step 3501), and when a pertinent document name is found (step 3502 yes), the obtained document ID is transferred to the control unit 21 (step 3503) and the time data for that timing is received from the timer unit 29 (step 3504).

Then, the retrieval unit 23 activates the communication unit 30 (step 3505), and transfer the time data to the communication unit 30 (step 3506). In response, the activated communication unit 30 transmits the user ID of the user who is operating the first device and the transferred time data. Here, the communication unit 30 transmits this information in a format shown in FIG. 36. Then, the retrieval unit 23 stores the time data and the document ID and document name of the retrieved document in the operation history storage unit 31 (step 3507). Here, the operation history storage unit 31 stores the data in a format shown in FIG. 37. Also, in a case a plurality of document data are retrieved, instead of storing the document IDs and the document names of all the retrieved document data, it is possible to store only the document ID of the document data which is selected by the user on the retrieval result display unit 22 to display its content.

In addition to the looking up and the correction of the already existing document as described above, the user can also produce the new document. In this case, the input unit 20 stores the input entered by the user into the document data storage unit 25 along with the document ID and the document name specified by the user, and transfers the document ID to the control unit 21. The control unit 21 then activates the retrieval unit 23 and transfers the document ID to the retrieval unit 23. In response, the retrieval unit 23 operates according to the flow chart of FIG. 38 as follows.

Namely, the retrieval unit 23 receives the document ID from the control unit 21 (step 3801), and the time data from the timer unit 29 (step 3802). Then, the retrieval unit 23 activates the communication unit 30 (step 3803), and transfers the time data to the communication unit 30 (step 3804). Finally, the retrieval unit 23 stores the time data and the document ID and document name of the produced document in the operation history storage unit 31 (step 3805).

In the second device, the processing in a case of receiving the information transmitted from an arbitrary one of the first devices, and the processing for retrieving the document looked up, corrected, or produced in the past by using the location name. Here, the communication unit 30 of the second device is assigned with the location ID indicating a location at which it is provided.

When an arbitrary communication unit 30-i receives the information transmitted from an arbitrary one of the first devices, the control unit 21 stores a set of the received information and the location ID assigned to that communication unit 30-i in the position operation information stora