Management server for synchronization system6757696Abstract A synchronization agent management server coupled to a network having a plurality of synchronization agents coupled thereto in communication with the management server, comprising a user login authenticator, a user data flow controller, and a unique user identification controller. The management server communicates with at least one interactive agent to control data migration between a computer to a network storage device. In one aspect, the management server is part of a data synchronization system for data between network coupled devices which includes a storage server having a data store of change transactions coupled to the network. Claims What is claimed is: Description LIMITED COPYRIGHT WAIVER
class CMyF1AppObj :
public Item,
public AppObj,
protected ModuleIdentity,
protected DataClassInfo,
protected ItemTypeInfo,
protected ItemFieldMap,
protected FolderInfo,
protected DataFileInfo,
protected SynchNotify,
protected ErrorMsg,
protected EnumItems,
protected FIndItem,
protected ModifyItem
{
public:
CMyAppObj ( HWND hWndParent );
.about.CMyFppObj (.sup. );
};
AppObj can contain children objects. They are Store objects. EnumItems interface is used to enumerate Store objects. FindItem interface is used to find the contained objects. ModifyItem interface enables AppObj to create a new Store object. AppObj is created by StructuredDelta calling CreateAppObject(HWND hWndParent, AppObj **ppObj). Store The Store object represents a database of the individual application information. If the individual application can handle multiple databases at same time, one needs multiple Store objects. One can think of Store object as a specialized Folder object, the root folder of each particular application data domain. The C++ example of Store's definition is shown below:
class CMyStore
public Item,
public ItemContainer,
protected EnumItems,
protected FindItem,
protected FindItemByData,
protected ModifyItem,
protected ReadWrite
{
CMyStore (.sup. );
.about.CMystore (.sup. );
};
Store is a container of Folder objects. EnumItems interface enables the enumeration of its contained folders while FindItem and FindItemByData interface is used to find contained Folders or Item objects. ModifyItem and ReadWrite interface enables the modification of each application database. Folder Folder object is a specific data class of each individual application such as a table in the relational database or a collection of data records in each application. For example, the applications contact collection can be thought as a Folder object. The C++ example of Folder's definition is shown below:
class CMyFolder :
public Item,
public ItemContainer,
protected EnumItems,
protected FindItem,
protected FindItemByData,
protected ModifyItem,
protected ReadWrite
{
public:
CMyFolder(.sup. )
.about.CMyFolder(.sup. );
};
The folder object is also a container. It can contain Item objects as well as Folder objects. EnumItems interface allows the enumeration of either Folder objects or Item objects or both. The FindItem and FindItemByData interface is used to find contained Folder objects or Item objects. The ModifyItem and ReadWrite interface enables the modification of an application's data tables. Item Item object represents an individual entity of each application's domain specific data. The Item object can be thought as a record of each application's relational table. For example, a contact, email, calendar, to-do item in the particular application can be thought of as an Item object. The C++ example of Item's definition is shown below:
class CMyItem :
public Item,
protected EnumItems,
protected FindItem,
protected ModifyItem,
protected ReadWrite
{
public:
CMyItem(.sup. );
.about.CMyItem(.sup. );
};
Item can contain Attachment objects only. EnumItems interface enables the enumeration of Attachment objects if any. ModifyItem and ReadWrite interface enables the modification of an application's records or data fields. Attachment The Attachment object is a specialized Item object that encapsulates an attachment data or relationship. Only the Item object can have Attachment objects. Attachment object can be thought as attachment data such as attached-email files. Attachment can also be thought as attachment relationship to other Item objects. The example of that is the distribution list (Item object) can contain contacts (Item objects). The C++ example of Item's definition is shown below:
class CMyItemAttachment :
public Item,
protected ReadWrite,
protected ModifyItem
{
public:
CMyItemAttachment (.sup. );
.about.CMyItemAttachment (.sup. );
};
Variant Variant object represents a data field of each particular application data. For example, a `first name` of a contact or the birthday date of a contact can be thought as Variant object. StructuredDelta only understands Variant object and the types of data fields it encapsulated. Variant object can contain any one of the following data field type:
struct Variant
{
enumFieldTag tag;
enumFieldDataFlag flag; // flags item fields as not
known or otherwise special
union
{
short int i; // eFieldType_WORD
LONG 1; // eFieldType_LONG
DWORD dw; // eFieldType_DWORD
unsigned_int64 qw; // eFieldType_QWORD
UUID uuid; // eFieldType_UUID
DATE time; // eFieldType_DATE
LPTSTR psz; // eFieldType_String
Binary bin; // eFieldType_Binary
Float flt; // eFieldType_Float
Double dbl; // eFieldType_Double
F1Collection coll; // eFieldType_Collection
} Value;
Stream* strm; // eFieldType_Stream
};
Variant::tag is an identification tag of data field and variant::flag specifies the type of data field while Variant::value member variable stores each application's field value. One data field type is Collection. Collection object is an array of Variant objects. It can be used to represent a compound data fields.
struct Collection
{
ULONG cValues;
struct_Variant** paVar; // This array really
contains cValues entries
};
Another exemplary data field type is Binary. A Binary object can be used to represent a binary data as it is.
struct Binary
{
ULONG cb;
LPBYTE 1pb;
};
AO Interfaces Each AO object has an AO COM interface. Each object must implement some of those interfaces to create certain capability or desired behavior that are expected by StructuredDelta. IItem This is the base interface of all application objects. It provides the identification service to StructuredDelta. Every object must have a unique ID, parent unique ID, display name, and item type information (eItemType.sub.- FOLDER, eItemType_CONTACT, etc). The unique ID is a unique string only in a given device. It is not persistent cross the Internet to other devices. The ID usually comes from the third party application database domain such a unique ID of a record.
Interface IItem : IUnknown
{
STDMETHOD_(LPCTSTR, GetUniqueID) (.sup. ) const PURE;
STDMETHOD_(LPCTSTR, GetParentUniqueID) (.sup. ) const PURE;
STDMETHOD_(LPCTSTR, GetDisplayName) (.sup. ) const PURE;
STDMETHOD_(enumItemType, GetItemType) (.sup. ) const PURE;
STDMETHOD_(BOOL, IsContainer) (.sup. ) const PURE;
STDMETHOD_(DATE, GetLastModificationTime) (.sup. ) const PURE;
STUMETHOD_(QWORD, GetSize) (.sup. ) const PURE;
STDMETHOD_(DWORD, GetFlags) (.sup. ) const PURE;
};
IItemContainer This is the base interface of all application container objects (store, folder). These container objects must have this interface implemented so that StructuredDelta would recursively descend in them if they have IItemContainer capability.
interface IItemContainer : IItem
{
STDMETHOD_(BOOL, ContainsItemType) ( enumItemType
eItemType ) PURE;
STDMETHOD_(BOOL, ContainsDataClass) ( enumDataClass
eDataClass ) PURE;
STDMETHOD_(enumSpecialFolderType, GetSpecialFolderType)
(.sup. ) PURE;
STDMETHOD_(GUID, GetMappingGUID) (.sup. ) PURE;
};
IErrorMsg This is an error-reporting interface for every application object. It is used by StructuredDelta to query the error string after a failure. The AO should implement this on every object after the error occurs and before returning the control to StructuredDelta.
interface IErrorMsg : IUnknown
{
STDMETHOD(GetErrorString) ( LPTSTR pszError, int iBufLen )
const PURE;
};
IEnumItems This is an interface for collection enumeration, used by StructuredDelta to enumerate the objects of the third party application database. IEItemEnumFlags (eItemEnumFlags_FOLDER, eItemEnumFlags_ITEM, and eItemEnumFlags_ATTACHMENT) is used to enumerate only the requested type of objects.
interface IEnumItems : IUnknown
{
STDMETHOD(ItemQueryStart) ( enumItemType type, long
&lCount, eItemEnumFlags dwFlags ) PURE;
STDMETHOD(ItemQueryNext) ( Item **ppItem ) PURE;
STDMETHOD(ItemQueryFinish) ( ) PURE;
};
IFindItem This is an interface for recursively finding object within the third party application database, used by StructuredDelta to find application object by its unique ID.
interface IFindItem : IUnknown
{
STDMETHOD(FindStoreByID) ( LPCTSTR pszUniqueID, ItemContainer
**ppFolder ) PURE;
STDMETHOD(FindFolderByID) ( LPCTSTR pszUniqueID, ItemContainer
**ppFolder ) PURE;
STDMETHOD(FindItemByID) ( LPCTSTR pszUniqueID, Item **ppItem )
PURE;
};
IFindItemByData This is an interface for recursively finding the object that matches the search criteria data. The search criteria are represented as Collection that allows the multiple search field keys to be used during the search. The multiple objects may be found that match the search criteria. The interface also provides enumeration capability of the search results.
interface IFindItemByData : IUnknown
{
STDMETHOD(FindByDataStart) ( enumItemType type, Variant*
pSearchKey, int* pnFound ) PURE;
STDMETHOD(FindByDataNext) ( LPTSTR pszEntryID, int
cbBufSize ) PURE;
STDMETHOD(FindByDataFinish) ( ) PURE;
};
IModifyItem This is an interface for StructuredDelta to add, delete, and re-parent application data in the third party database during synchronization.
interface IModifyItem : IUnknown
{
STDMETHOD(Add) ( BOOL bFolder, enumItemType type, Item
**ppItem ) PURE;
STDMETHOD(Delete) ( ) PURE;
STDMETHOD(Move) ( ItemContainer * pDestFolder ) PURE;
};
IReadWrite This is an interface for accessing, writing, and mapping the third party application data fields by StructuredDelta. It provides the capability of read and write data fields from and to the third party application database and the capability of mapping data field of the third party application to universal data format of the system of the present invention. Any object that has data fields and require field level synchronization must implement this interface.
interface IReadWrite : IUnknown
{
STDMETHOD(Read) ( ) PURE;
STDMETHOD(Commit) ( ) PURE;
STDMETHOD(GetFieldData) ( enumFieldTag fieldTag, Variant
**ppVariant ) PURE;
STDMETHOD(ReleaseFieldData) ( Variant *pVariant ) PURE;
STDMETHOD(SetFieldData) ( const Variant *pVariant ) PURE;
};
IAppObj This is an AppObj only interface. It provides the capability of logon and logoff to the third party applications during synchronization. The data class filter mechanism is used by StructuredDelta to filter the enumeration of contained data classes (eDataClass_CONTACT, eDataClass_CALENDAR, etc).
interface IAppObj : IUnknown
{
STDMETHOD(Logon) ( HWND hWndParent ) PURE;
STDMETHOD(Logoff) ( ) PURE;
STDMETHOD(SetFilter) ( const VOID* pFilter, int BufLen )
PURE;
STDMETHOD_(int. GetFilter) ( VOID* pFilter, int BufLen )
PURE;
};
IModuleIdentity This is an AppObj only interface. It provides DLL module identification information to the Manager object such as the name of the third party application, enum ID of this application, and the application installation detection support.
interface IModuleIdentity : IUnknown
{
STDMETHOD(GetName) ( LPTSTR pszName, int iBufLen ) const
PURE;
STDMETHOD(GetAppl) ( Appl *pAppl ) const PURE;
STDMETHOD(IsInstalled) ( BOOL *bIsInstalled ) const PURE;
};
IItemTypeInfo This is an AppObj only interface. It provides the information on the number of item types supported by AO, what type items are supported and the capabilities for a specific item type. This returns a DWORD containing bits set.
interface IItemTypeInfo : IUnknown
{
STDMETHOD(GetSupportedTypesCount) ( int &iCount ) PURE;
STDMETHOD (GetSupportedTypeInfo) ( int iIndex, enumItemType
&type, LPTSTR pszTypeName, int iBufLen ) PURE;
STDMETHOD(GetItemTypeCaps) ( enumItemType type, DWORD
&dwFlags ) PURE;
};
IDataClassInfo This is a CAppObj only interface. It provides the information on the number of data classes that are supported by the application object and what the data classes are supported
interface IDataClassInfo : IUnknown
{
STDMETHOD(GetCount) ( int *piCount ) PURE;
STDMETHOD(GetDataClass) ( int Index, enumDataClass
*peDataClass ) PURE;
};
IDataFileInfo This is a CAppObj only interface, it provides information on the number of database files and database filenames supported by AO to avoid being synched twice by application sync and file-set sync.
interface IDataFileInfo : IUnknown
{
STDMETHOD(GetDataFileCount) ( int *piCount ) PURE;
STDMETHOD(GetDataFilePath) ( int iIndex, LPTSTR
pszFilePath, int iBufLen ) PURE;
} ;
IItemFieldMap This is a CAppObj only interface that is used by StructuredDelta to query the data fields of given application object. For example, what are data fields in application object called eItemType_CONTACT?
interface IItemFieldMap : IUnknown
{
STDMETHOD(FieldQueryStart) ( const enumItemType &type, int
&iCount ) PURE;
STDMETHOD(FieldQueryNext) ( enumFieldTag &field, LPTSTR
pszName, int iBufLen, LPTSTR pszType, int iTypeBufLen ) PURE;
STDMETHOD(FieldQueryFinish) (.sup. ) PURE;
} ;
IFolderInfo This is a CAppObj only interface, used by StructuredDelta to obtain the special and default folders' unique IDs and UUIDs.
interface IFolderInfo : IUnknown
{
STDMETHOD(GetSpecialFolderID) ( enumSpecialFolderType
eFolder, LPTSTR pszUniqueID, int iBufLen ) PURE;
STDMETHOD(GetDefaultFolderID) ( enumItemType type, LPTSTR
pszUniqueID, int iBufLen ) PURE;
STDMETHOD(MapFolderGUID) ( UUID uuidFolder, LPTSTR
pszUniqueID, int iBufLen ) PURE;
} ;
IFastSync This is a CAppObj only interface that is used by StructuredDelta to query if the given AO also provides FastSync service or not. FastSync is a DLL component that is written using the third party APIs and loaded into the third party application to receive the changes in database while users are operating the application. It is used to speed up the synchronization performance by syncing only the objects that are known to IFastSync component.
interface IFastSync : IUnknown
{
STDMETHOD(GetFastSync) ( enumDataClass eDataClass, BOOL*
pbFastSync ) PURE;
} ;
SynchNotify This is a CAppObj only interface that is called by Manager to notify the third party application the state of synchronization: start, finished, or reset so that the application can prepare itself accordingly.
interface ISynchNotify : IUnknown
{
STDMETHOD(SynchNotify) ( enumSynchNotify eNotify ) PURE;
} ;
Server AO Server Application Objects share many characteristics with desktop application objects, including support for reading and mapping to the universal record structure set forth above. Nevertheless, among various devices incorporated into the system of the present invention, each application object database will be quite different. For example, the Palm.RTM. database on the device is really just a memory space with records laid out sequentially in memory. In a web portal-type application, the application object may be an Oracle database. Server application objects may generally have less difficult tasks since the applications supported are generally either devices providing their own change data output, (such as Palm.RTM.-type PDA's), or which do not have a great deal of data to export (such as cell phones, having only name and number information). Nevertheless, each application object must support all calls defined in a class interface definition as follows:
FUNCTION DESCRIPTION
Open Perform an initialization of the device before
data retrieval functions are called.
Close Done with database calls, cleanup if
necessary.
Get First Modified Record Get the first modified record from the device
and insert into application object.
Get Next Modified Record Get the next modified record from the
device and insert into the application object.
Add Record Add a record into the application object
database.
Update Record Update a record.
Delete Record Delete a record in the application object
database.
Set Device Records A function called during the synchronization
manager to send a bytestream to the
application object for interpretation. The
bytestream will contain a list of records to
add to the application object modified
records list. At a later point in time, such
records will be retrieved by the Get First
Modified Record/Get Next Modified Record
functions.
Get Device Records For records bound to the device, this call
gets a bytestream that contains a list of
records to add back to the device. There is
an outbound record list that is saved until
this call is finished, at which time the sync
server will be finished with the application
object.
Set Device Response A function used to modify or repair a record
input saved in the application object store
that was sent to the device in the Get
Device Records call, such as a record ID for
a record. If 10 records were sent to the
device during the Get Device Records call,
one would expect to see 10 records coming
back in during this function call.
As noted above, because each application object database is different, the calling convention and the application object itself will likewise be different. The calling convention for a Palm.RTM. device's sync manager application object is given in the following pseudo-code:
Call AO::Open
Call AO::WriteRecords
Start synchronization process
While more records in AO Data Object
Call AO::GetFirstModifiedRecord(.sup. )
Call AO::GetNextModifiedRecord(.sup. )
END
IF new records THEN
Call AO::AddRecord(.sup. )
IF deleted records THEN
Call AO::DeleteRecord(.sup. )
IF update record THEN
CALL AO::UpdateRecord(.sup. )
Call AO::Close
As shown therein, the calling convention is designed to be integrated with the Palm's.RTM. own sync manager. A second example provided below shows mapping of a portion of a web portal's contact database:
MappingItem CContactTable::m_FieldMap [.sup. ] =
{
{1, eFieldTag_Contact_FirstName, "firstname"} ,
{1, eFieldTag_Contact_MiddleName, "middlename"} ,
{1, eFieldTag_Contact_LastName, "lastname"} ,
{1, eFieldTag_Contact_Title, "title"} ,
{1, eFieldTag_Contact_Suffix, "suffix"} ,
{1, eFieldTag_Contact_Anniversary, "anniversary"} ,
{1, eFieldTag_Contact_Birthday, "birthday" } ,
{1, eFieldTag_Contact_AssistantName, "assistantname"} ,
{1, eFieldTag_Contact_Children, "children"} ,
{1, eFieldTag_Contact_CompanyName, "companyname"} ,
{1, eFieldTag_Contact_Department, "department"} ,
{1, eFieldTag_Contact_FTPSite, "ftpsite"} ,
{1, eFieldTag_Contact_Gender, "gender"} ,
{1, eFieldTag_Contact_JobTitle, "jobtitle"} ,
{1, eFieldTag_Contact_ManagerName, "managername"} ,
{1, eFieldTag_Contact_NickName, "nickname"} ,
{1, eFieldTag_Contact_Office, "office"} ,
{1, eFieldTag_Contact_Profession, "profession"} ,
{1, eFieldTag_Contact_Spouse, "spouse"} ,
{1, eFieldTag_Contact_SelectedMailingAddress,
"selectedmailingaddress"}
} ;
int CContactTable::m_nNumFields =
sizeof(m_FieldMap)/sizeof (MappingItem) ;
HRESULT CPortalAddrOCI::InsertRecord( MappingItem theMap [.sup. ], int
numFields, CDataAccessor *pInsertItem, CF1ItemUniversal *pUnivItem,
bool bForceCreate )
{
bool bHasData = SetRecordFields( theMap, numFields, pInsertItem,
pUnivItem ) ;
if( bHasData .vertline..vertline. bForceCreate )
{
// Insert the record into the database and execute the command
pInsertItem->InsertRow(0) ;
pInsertItem->Exec (.sup. ) ;
}
return S_OK;
}
The above example of mapping the contact field files maps contact fields from a particular web contact information database to fields in the universal record format from the master list header file (pio_types.h) in the system of the present invention. This mapping is for a specific contact table and it should be understood that other information, such as phone numbers, e-mail addresses, and other contact information may be stored in a separate table. Once data is extracted from a particular application, the server application object must then convert the information into the universal record format which can be utilized by other server device engines to take content information into their own particular application. Universal Record Format The universal record format is used by each server device engine to handle various tasks of encapsulating records in a common format, comparing records, creating and holding differences between records, and other tasks of synchronization. The universal record format allows the application objects to support a wide range of extensible application item types such as contacts, calendar, mail, bookmarks, and the like. Flexible type name and value associations permit synchronization without regard to individual vendor application information formats. Each application object encapsulates mapped knowledge from the vendor unique format to the universal format of the present invention. As such, an application object can be designed to support any combination of application and binary information types. In essence, application objects can be designed to support a vendor application using only binary file synchronization if the internal format of the application is not known. Server application objects can also be designed to create collections. For example, if the user wishes to create a "my pictures" collection which consists of some collection of information and synchronize this collection of information, such an arbitrary grouping of classes of information into appropriate representations is supported. Because the connector layer of the interfaces to the actual storage with a vendor application varies with application type, application access methods can include, but are not limited to, disk or database access, network protocols, wireless device protocols, and the like. The Universal Records Format and the Universal Field Format class definitions are given below:
typedef map < enumFieldTag, CUniversalField, less_enumFieldTag >
UniversalRecordMap;
typedef UniversalRecordMap::value_type UniversalRecordPair;
typedef UniversalRecordMap::iterator UniversalRecordIterator;
typedef UniversalRecordMap::const_iterator
ConstUniversalRecordIterator;
class CUniversalRecord
{
private:
UniversalRecordMap recordMap_;
public:
bool conflicts(const CUniversalRecord& rhs) ;
bool add(const CUniversalRecord &rhs) ;
bool subtract(const CUniversalRecord& rhs) ;
CUniversalRecord(.sup. ) ;
CUniversalRecord( const CUniversalRecord& rhs ) ;
virtual .about.CUniversalRecord(.sup. ) ;
// add this element
HRESULT insert( enumFieldTag eId, long value,
enumFieldDataFlag flag = eFieldDataFlag_Normal) ;
HRESULT insert( enumFieldTag eId, LPCTSTR value,
enumFieldDataFlag flag = eFieldDataFlag_Normal) ;
HRESULT insert( enumFieldTag eId, DATE value,
enumFieldDataFlag flag = eFieldDataFlag_Normal) ;
HRESULT insert( enumFieldTag eId, string value,
enumFieldDataFlag flag = eFieldDataFlag_Normal) ;
HRESULT insert( UniversalRecordPair p ) ;
CUniversalRecord exclusiveLeftWins( CUniversalRecord& rhs ) ;
CUniversalRecord inclusiveLeftWins( CUniversalRecord& rhs ) ;
bool removeSame(const CUniversalRecord &rhs) ;
bool Find( const enumFieldTag eId, CUniversalField &field ) ;
UniversalRecordMap::iterator find( const enumFieldTag eId ) {
return recordMap_.find(eId) ; }
UniversalRecordMap::iterator begin(.sup. ) { return
recordMap_.begin(.sup. ) ; }
UniversalRecordMap::iterator end(.sup. ) { return
recordMap_.end(.sup. ) ; }
bool empty(.sup. ) { return
recordMap_.empty(.sup. ) ; }
long size(.sup. ) { return
recordMap_.size(.sup. ); }
UniversalRecordMap::iterator erase (UniversalRecordMap::iterator&
it)
{ return recordMap_.erase(it) ; }
void clear(.sup. ) { recordMap_.clear(.sup. ) ; }
} ;
The UniversalField Structure
class CUniversalField
{
public:
enum eUniversalField
{
eUF_Unknown,
eUF_Long,
eUF_String,
eUF_Date,
eUF_Blob
};
protected:
eUniversalField typeId_;
enumFieldTag fieldId_;
enumFieldDataFlag flag_;
size_t len_;
union
{
long l;
DATE d;
TCHAR* pCh;
} value_;
public:
CUniversalField(.sup. ) ;
CUniversalField( const CUniversalField& rhs ) ;
CUniversalField( enumFieldTag itemId, long value,
enumFieldDataFlag flag = eFieldDataFlag_Normal) ;
CUniversalField( enumFieldTag itemId, DATE value,
enumFieldDataFlag flag = eFieldDataFlag_Normal) ;
CUniversalField( enumFieldTag itemId, LPCTSTR value,
enumFieldDataFlag flag = eFieldDataFlag_Normal) ;
CUniversalField( enumFieldTag itemId, string blob,
enumFieldDataFlag flag = eFieldDataFlag_Normal) ;
.about.CUniversalField(.sup. ) ;
bool operator==( const CUniversalField& rhs ) const;
bool operator!=( const CUniversalField& rhs ) const { return
!operator==(rhs) ; }
CUniversalField& operator=( const CUniversalField& rhs) ;
eUniversalField getType(.sup. ) const { return typeId_; }
enumFieldTag getFieldID(.sup. ) const { return fieldId_; }
enumFieldDataFlag getFlag(.sup. ) const { return flag_; }
size_t getLength(.sup. ) const { return len_; }
LPCTSTR getString(.sup. ) const { ASSERT( eUF_String ==
typeId_) ; return value_.pCh; }
long getLong(.sup. ) const { ASSERT( eUF_Long ==
typeId_) ; return value_.l; }
DATE getDate(.sup. ) const { ASSERT( eUF_Date ==
typeId_) ; return value_.d; }
string getBlob(.sup. ) const { ASSERT( eUF_Blob ==
typeId_) ; return string(value_.pCh,len_) ; }
void get( LPCTSTR& p ) const { ASSERT( eUF_String ==
typeId_) ; p = value_.pCh; }
void get( long& l) const { ASSERT( eUF_Long ==
typeId_) ; l = value_.l; }
void get( DATE& d) const { ASSERT( eUF_Date ==
typeId_) ; d = value_.d; }
void get( string& b) const { ASSERT( eUF_Blob ==
typeId_) ; b.assign(value_.pCh,len_); }
bool isString(.sup. ) const { return typeId_ ==
eUF_String; }
bool isLong(.sup. ) const { return typeId_ == eUF_Long; }
bool isDate(.sup. ) const { return typeId_ == eUF_Date; }
bool isBlob(.sup. ) const { return typeId_ == eUF_Blob; }
} ;
EXAMPLE An example of how data is removed from one particular application data type and converted into the universal record format is given below for an Oracle database:
#include "stdafx.h"
#include <string>
using namespace std;
#include "F1ItemUniversal.h"
#include "oci.h"
#include "OCIDefs.h"
#include "OCIConnect.h"
#include "OCISession.h"
#include "OCIColumn.h"
#include "OCICursor.h"
#include "DataAccessor.h"
#include "UniversalMapper.h"
#include "UniversalRecord.h"
#include "F1Util.h"
#include "BaseAOSTableOCI.h"
/*
* Function: MapFields
* Description: Map fields from an Oracle database record into an
UniversalRecord format.
*/
void CBaseAOSTableOCI::MapFields( CDataAccessor *pAccessor,
MappingItem theMap[.sup. ], int numFields, CUniversalRecord &univRec )
{
string sValue;
DATE dtValue;
LONG lValue;
double dValue;
for( int inx=0; inx<numFields; inx++ )
{
enumFieldTag fieldID = theMap[inx] .m_universalFieldID;
switch( F1PropType( fieldID ) )
{
case eFieldType_Binary:
{
// to fill properly, 1st name and last name should already be assigned
CUniversalField emailField;
string sValue;
if ( SUCCEEDED(BuildEmailField( pAccessor,
sValue, emailField )) )
univRec.insert( fieldID,
emailField.getBlob(.sup. ) ) ;
break;
}
case eFieldType_String:
if( pAccessor->GetFieldValue( fieldID, sValue )
)
{
if ( 0 == ::_tcslen(sValue.c_str(.sup. )) )
continue;
univRec.insert( fieldID, sValue.c_str(.sup. )
) ;
}
break;
case eFieldType_DATE:
if( pAccessor->GetFieldValue( fieldID, dtValue
) )
univRec.insert( fieldID, dtValue ) ;
break;
case eFieldType_DWORD:
if( pAccessor->GetFieldValue( fieldID, lValue )
)
univRec.insert( fieldID, lValue ) ;
break;
case eFieldType_Double:
if( pAccessor->GetFieldValue( fieldID, dValue )
)
univRec.insert( fieldID, dValue ) ;
break;
}
}
}
HRESULT CBaseAOSTableOCI::InsertRecord( MappingItem theMap[.sup. ],
int numFields, CDataAccessor *pInsertItem, CF1ItemUniversal
*pUnivItem, bool bForceCreate )
{
bool bHasData = SetRecordFields( theMap, numFields, pInsertItem,
pUnivItem ) ;
if( bHasData .vertline..vertline. bForceCreate )
{
pInsertItem->InsertRow(0) ;
pInsertItem->Exec(.sup. ) ;
}
return S_OK;
}
/*
* Function: SetRecordFields
* Description: Map fields from an UniversalRecord format into an
Oracle database record (pInsertItem)
*/
bool CBaseAOSTableOCI::SetRecordFields( MappingItem theMap[.sup. ], int
numFields, CDataAccessor *pInsertItem, CF1ItemUniversal *pUnivItem )
{
bool bHasData = false;
CUniversalField field;
for( int inx=0; inx<numFields; inx++ )
{
enumFieldTag fieldID = theMap[inx].m_universalFieldID;
BOOL bExists = pUnivItem->m_record.Find( fieldID, field ) ;
if( bExists )
{
bHasData = true;
if( field.isBlob(.sup. ) )
{
string blob = field.getBlob(.sup. ) ;
LPCTSTR szEmailAddr =
GetAddressFromRecipients((F1RECIPIENTS*)blob.c_str(.sup. )) ;
if( szEmailAddr && *szEmailAddr != NULL )
pInsertItem->SetFieldValue( fieldID,
(string)szEmailAddr ) ;
else
{
bHasData = false;
continue;
}
}
else if( field.isString(.sup. ) )
{
string sValue = field.getString(.sup. ) ;
if( !sValue.empty(.sup. ) )
pInsertItem->SetFieldValue( fieldID,
sValue ) ;
}
else if( field.isLong(.sup. ) )
{
LONG nValue = field.getLong(.sup. ) ;
pInsertItem->SetFieldValue( fieldID, nValue) ;
}
else if( field.isDate(.sup. ) )
{
DATE dValue = field.getDate(.sup. ) ;
if( dValue )
pInsertItem->SetFieldValue( fieldID,
dValue ) ;
}
} // if( bExists )
} // For all fields
return bHasData;
}
While the above-identified code is specific to, for example, an Oracle database, one of average skill in the art will readily recognize that the technique utilized above may be adapted to other types of databases containing records and fields of interest. In the above code examples, all fields which are mapped from a particular application are mapped to fields in the master mapping file. Management Server In order to provide security and identification of particular users in an Internet-implemented synchronization system, a management server may be provided in the system of the present invention. The management server is a centralized server which controls behavior and characteristics of the entire network of device engines across all users. FIG. 14 shows the general representation of how a management server 1410 integrates itself into the system of the present invention. Shown in FIG. 14 is an exemplary device engine 1450 which has HTTP links to both a management server 1410, a storage server 1415, and a generic FTP server 1420. As will be discussed hereinafter with reference to the process of the present invention, and the specific implementation of the data below shown in FIGS. 15-17, the management server interacts with the device engine to control authorized access to information on the storage server, or a generic FTP server 1420, 1425 to access device-specific information storage 1430 in accordance with the system of the present invention. This allows any device coupling to the Internet to have access to management protocols and to retain user information across all platforms which the data which is being synched by the system of the present invention must access. The management server communicates using hypertext transfer protocol (HTTP) which may be implemented with a secure sockets layer (SSL) to ensure security. In particular, the management server supports an authentication interface that requires each device engine to authenticate with the management server before performing synchronization. Certain storage server implementations may utilize locking semantics to control read and write access to storage for multiple device engines. For example, in a generic FTP request, if two device engines attempt to connect to the same data at the same time, there must be some form of locking control to prevent device engines accessing the same data at the same time. In this instance, the management server controls the device engine acquisition, renewal, and releasing of locks against data stored in the network. Each device engine is uniquely identified and tracked by the management server. This allows for tailoring behavior between the management server and specific types of storage systems and device engine components. All device engine components are tagged and version stamped for management via the management server. Device actions can request updated copies of individual device engine components, permitting self-update and configuration of device engine systems. This permits minimal download designs for device engines that are on low bandwidth connections enabling the device engines to download additional required components at a later time. In a further aspect of the system, a value added component may be provided where the management server can support client's advertising mechanisms, enabling the display of banner or similar advertising on a device engine system without the need for a web browser. Cycling of advertisements, statistic collection, and the like, are managed via management server protocols. Online purchase and subscription mechanisms are also supported using the management server protocol. The management server further supports the accounting, sign-up registration, device edition, storage server selection, and similar functions for each user in the system. In one embodiment, the management server may retain password and encryption information for a given user account. In a second embodiment, such information is not retained. The second embodiment provides the advantage that users may feel more secure if the maintainer of the management server is not in possession of the password to access data in the user's account. Further information with respect to the management server and the data flow from the management server to other components of the system of the present invention will become apparent with respect to the discussion of the process flow and data flow diagrams in FIGS. 15-17. FIG. 17 shows a general depiction of the data flow and the functional specification of the management server utilized in accordance with the present invention. As shown in FIG. 17, following a welcome request 1710, a user is allowed to sign out which enables an add user module 1712, and subsequently enables an add device module 1714. If sign-up is not requested, information may be provided via module 1718. As indicated in FIG. 17, the add user module 1712 adds user records to the user in device database 1750. Additionally, the add device module 1714 adds users and devices to the user device database 1750. A device list 1720, and a device engine download and update database 1722, provide selection data for the add device module 1714. The account authentication module 1724 receives input both directly from a user log-in from the welcome screen at 1710 and from the add device module 1714. Once an account is authenticated and confirmed, the administrator of the system of the present invention having a private data store at 1770 may choose to provide a web desktop 1754 which allows access to a user's records such as file 1756, e-mail 1758, calendar 1760, contacts 1762, notes 1764, and tasks 1766. The information will be culled from a provider database 1752 which will be synched in accordance with the system of the present invention as previously described. In essence, the provider database 1752 accesses data from the device engines 1780, which include, as discussed above, the storage server, each individual device engine 1785, and a settings database 1787. Other portions of the management server include the locking modules for beginning a sync 1732, continuing a sync 1734, and ending a sync 1736, and for updating user information including modifying a user 1742, adding devices 1744, removing devices 1746, and modifying devices 1748. Storage Server Shown in FIG. 14 is the storage server 1415. While storage server 1415 may include a generic storage model accessible through any number of standard Internet protocols, in accordance with the present invention, a flexible storage architecture is provided that permits various standard implementations of the system of the present invention. This allows deployment of network services without installation of new server applications and can be responsible for communicating change information between multiple device engines in a consistent fashion. One or more storage servers 1415 may be used to communicate transaction amongst a collection of devices. Each user's personal information network is represented by a unique account within its own data package storage section. The storage server 1415 maintains persistent store collection of data packages which is, at a minimum, enough data packages to be capable of synchronizing the most out-of-date system in a user's given information network or add information to new devices which are provided in the network. Additional data packages can be maintained to permit rollback of previous versions of information. The storage server can automatically dispose of older data package storage and can support aging of an inactive accounts. Each storage server 1415 may be implemented using a variety of implementations including a standard FTP server for any operating system platform. The storage server can be implemented using HTTP protocols for increased efficiency and firewall avoidance. The storage server may be implemented using techniques for local storage such as database access or single file storage of a user's entire file system tree. The storage server 1415 may utilize the stored foreign protocol model for moving copies of data packages to other storage servers in the system. In one embodiment, the storage server can allow tunneling of information using an alternative protocol to other storage servers in cases where firewall prevents originating protocol. For example, a storage server can relay an FTP traffic inside an HTTP protocol. Storage servers may include their own locking semantics to arbitrate multiple device engine access to the same server without the need for a separate management server. Each device engine can access only a specific user's data package storage area even though the storage server 1415 may maintain a larger number of data packages across a large number of users. This allows for increased scaling when the storage server is implemented using file system techniques. In one aspect, the storage server is implemented using standard FTP or HTTP connections for each operation. HTTP is composed of request response pairs. All requests are supposed to be posting commands. Parameters can be set in the form known as "application/X-WWW-form-URLENCODED". The encoding is specified as in RFC1866. Functions for the storage server include testing if the storage server can reach other users which will retrieve a simple text string, a "get" command which transfers the contents of a file as in a binary stream of byes; a put command as a binary stream of data to the storage server, a directory listing command, a remove command, a rename command, an exist command, and the like. Pull Synchronization FIG. 15 represents a "pull" synchronization process in accordance with the present invention. Both the pull synchronization illustrated in FIG. 15 and the push synchronization illustrated in FIG. 16 are done from the perspective of the device engine. A pull synchronization as illustrated in FIG. 15 is always performed prior to a push synchronization. This allows the device engine to know whether synchronization of its own data is necessary. Each device has its own triggering mechanism for initiating synchronization. Some devices, such as Windows clients and Palm.RTM. pilots are triggered manually when the user presses a "sync" button. Other devices, such as a cellular telephone, may be triggered automatically after another device completes a sync. Regular, time-based triggers are supported as well. A web-based application portal will sync when a user logs into the website security authorization mechanism, and may optionally sync on a log-out of the user or on the session time-out, but only if the user has changed data during the session. For each sync, the triggering event specifies which application types are to sync for the device. This enables a triggering event to trigger only a sync for a particular application type. The management server can specify that no sync is needed for a particular type of application to minimize traffic to the storage server. Syncs may be triggered via an HTTP request to the server. This request holds information about which device to sync and the user log-in information is bounced to the management server for authorization and validation. Syncs may be triggered by sending an HTTP request to the server and passing the authentication information in the data portion of the request to the management server. Each device may include a servlet that is responsible for retrieving the request and ensuring its proper format before passing the synchronization request on to the server. The device name and device class uniquely identify a particular device type that is being synchronized, and is contained in the management server. Each user has one or more device entries in the management server authorization records and each device name is unique for this user's space. For example, if a user has five devices with his or her own personal identification number, there will be five authorization records. There may be two Windows devices, two different Palm.RTM. devices and a web service portal, each having their own personal identification number. As shown in FIG. 15, the pull synchronization process starts at an idle state 1405 when the triggering event, described above, triggers a synchronization request. The synchronization request is confirmed at 1410 and if the request is verified, a connection is made to the storage server at step 1415. Once a connection is established, the connection to the management server is made at step 1420 to authenticate the user identification via the management server. If authentication is successful, the management server may initiate a management server lock on the storage server so that no conflicting device engines may couple to the same data at the same time. A failure at any of the steps 1410-1425 will return the system to its idle state 1405. Once the engine server lock is acquired, the storage server will be checked to determine whether a new version of the data exists on the storage server at step 1430. If no new version exists, the synchronization process ends. If a new version of the data exists, the device engine will retrieve the difference information at step 1435 "to get .DELTA.." Once a .DELTA. is retrieved, conflicts are resolved at step 1450. The resolve conflicts step allows a user to resolve conflicts to multiple types of data which have been changed on both the server portion of the device and in the local data. Once the conflicts have been resolved at step 1450, the .DELTA.'s are applied at step 1455. The apply .DELTA. step 1455 allows for filters and mappings to be accounted for on the local device engine side of the system. As shown at steps 1460, 1465, 1470, and 1475, the .DELTA. may include updates at the item level 1460, application level 1465, device level 1470, or network level 1475. In each of the aforementioned steps, a loop back to the .DELTA. retrieval step 1435 is provided. When no further .DELTA.'s are available, the management server lock is released at step 1440. The foregoing description of a pull synchronization is further described in the following pseudo-code:
*SymbolicSyncEngine::Sync
Download Remote File System
For each Symbolic app in the file system's list of symbolic apps
CFDESymbolicSyncEngine::SyncSymbolicApp
Create a structured delta object -- CStructuredDelta
delta( . . . )
Compare local and remote versions of deltas SODs),
if not the same then
while localVersion != remoteVersion
download remote version
// apply delta (change log)
delta.ApplyChangeLog
// See details below
increment local version
end while
else
nothing to do
end if
if any local items (change logs) are unsent then
delta->ApplyUnsentItems (Reads the changes [JCL
where applied?]
end if
// Generate a new change log for the device:
delta->delta.GenerateChangeLog( . . . strChangeLogFile
. . . ) // See details below
FTP it back up to Storage Server
Update the local version number
end // SymbolicSyncEngine::SyncSymbolicApp
end // CFDESymbolicSyncEngine::Sync
CStructuredDelta::ApplyChangeLog
Set up m_pAppObj; // IFAO pointer
Set up m_pAOS; // IAOS pointer
Other set up (statistics, time to complete, etc.)
Read the change log and . . .
ApplyChangeListToAOS(f1Changes)
for each itme in list
ApplyItemToAOS // (Does
m_pAOS . . . AddRecord/UpdateRecord/DeleteRecord)
end for
end // ApplyChangeListToAOS
If not doing a full sync, also add changes from this file to apply
these to m_F1Changes
end //CStructuredDelta::ApplyChangeLog
CStructuredDelta::GenerateChangeLog
Set up m_pAppObj; // IFAO pointer
Set up m_pAOS; // IAOS pointer
Other set up (statistics, time to complete, etc.)
// Set up m_deviceChanges by call to:
CStructuredDelta::CreateDeviceChangeList
Create a CF1Item* pItem
// Iterate FAO modifications:
for (m_pAppObj->GetFirstModified(pItem),
m_pAppObj->GetNextModified(pItem))
cast pItem to --> CF1ItemUniversal* pUniItem
// Do certain things based on whether the operation is
an add, delete, or update.
// Then in each case, call:
CStructuredDelta::GetMatchingItemFromAOS
// First get by F1ID
m_pAOS->GetRecordByF1ID
// See if we have an AppID, if so:
m_pAOS->GetRecordByAppID
// If we can build search key on it
iterate m_pAOS->GetFirstMatchingRecord /
m_pAOS->GetNextMatchingRecord
end // CStructuredDelta::GetMatchingItemFromAOS
end for
end // CStructuredDelta::CreateDeviceChangeList
if m_deviceChanges is not empty
// reconcile (compare) change lists while writing to AOS
CStructuredDelta::ReconcileChangeLists
For each item in m_deviceChanges . . .
If we can find it in m_F1Changes
Reconcile the device item and the f1
item
end if
ApplyItemToAOS // (Does
m_pAOS. . . AddRecord/UpdateRecord/DeleteRecord)
end for
end // CStructuredDelta::ReconcileChangeLists
// Create a new change log (F1 delta package)
ApplyChangeListToF1(m_deviceChanges)
m_deviceChange.Store
// Fires off its own whole world, see
F1ItemList.cpp
end // m_deviceChange.Store
end // ApplyChangeListToF1(m_deviceChanges)
report stats
end if
// Switch (SyncMode)
If SyncMode == full
ApplyAOSToDevice
iterate m_pAOS->GetFirstRecord . . .
m_pAOS->GetNextRecord
Add or update the corresponding FAO record.
(Note. Never delete based on what's in AOS, apparently) .
end // ApplyAOSToDevied
else
ApplyChangeListToDevice(m_f1Changes) ;
End // CStructuredDelta::GenerateChangeLog
Push Synchronization FIG. 16 shows a push synchronization in accordance with the system and method of the present invention. Beginning at idle state 1505, a synchronization event occurs and if confirmed at step 1510, .DELTA.'s are checked at step 1515. Depending on which type of changes occurred, a network .DELTA. 1520, device .DELTA. 1525, location .DELTA. 1530, or item .DELTA. 1535 will be created. Once the .DELTA.'s for a given application have been created, the method of the present invention continues at step 1540, which enables a connection to a storage server. Upon connection to the storage server, a further connection to management server 1545 will occur to authenticate the user in the system. Failure at any of the aforementioned points will result in returning to idle state 1505. Upon authentication, a management server lock is enabled to ensure that multiple device engines do not connect to the same data at the same time. Once a lock is acquired at step 1555, .DELTA.'s are uploaded to the system. As shown, this may include uploading an item .DELTA. 1575, an application .DELTA. 1570, uploading a device .DELTA. 1565, or a network .DELTA. 1560. Once .DELTA.'s have been uploaded to the server, management lock server 1580 is released, and the connection to the storage server is terminated at step 1585. It should be recognized that such a push synchronization need not occur directly to a server, but may occur directly to a second device engine in accordance with the depiction of the multiple embodiments of the invention in FIGS. 1-7. Data Package Specification Once information is provided into the universal data format, the device engine organizes the format into a data package. Each data package thus includes a description of changes to any and all information for particular application, and a collection of data packages describes changes across all device engines including all different types of data. With encoding and compression, data packages can become very compact to minimize bandwidth and storage requirements across the system of the present invention. In one particular aspect of the present invention, encoding of the data packages may be provided in a streaming format to allow processing by the device engines with minimal storage and memory configuration at the device engine level. The device engine can read the stream and determine which records from which applications it needs to update the particular information present on the system on which it resides. Data packages can be provided in a binary data format. This allows data packages to encode changes to non-application data at a byte level. Hence, if a single bit on a system changes, the system of the present invention allows synchronization of that bit on another system. Changes are described as a sequence of byte-level change operations. One such encoding is using a sequence of insert and copy operations. Insert and copy operations generally define a particular "insertion" of a number of bytes from a source file, then how many bytes of a changed source file must be inserted to a particular file, then how many bytes to insert from a particular new file, with a differencing engine taking the bytes in the stream and inserting them into the new file to create the new version of the file. As will be readily understood by one of average skill in the art, this allows a user to, for example, change a binary file such as a word processing document or other type of attachment, and synchronize such an attachment at the binary level. Specifically, if one forwards an e-mail of a word document to a second individual, the second individual modifies it and wishes to return this document with modifications to the first individual, because the first individual has the original file on his system, if both systems are enabled in the system of the present invention, the second system need only send the changes or the difference information back to the first system in order for the first system to reconstruct the document on the second system using this change data to create the document as intended by the second user. Multiple caching of both the generation and application of data packages can be utilized to deal with communication issues in accordance with the system of the present invention. It should be further recognized that data packages can be merged into larger meta-data packages. Such meta-data information, such as the organization of multiple device packages, may be encoded into a larger system package. Each system package is essentially an encoded sequence of data packages. FIG. 12 shows the general format of the data package and universal data format an object stream hierarchy used in accordance with the present invention. With reference to FIGS. 11 and 12, one will note that each item in a particular application data structure will have a particular classification, such as a file, folder, contact, e-mail, calendar, etc. as shown in FIG. 13. The universal data structure contains a mapped item field for each type of data possible from each application supported by the system. Hence a "master" list of | ||||||