scispace - formally typeset
Search or ask a question
Journal ArticleDOI

Interval-based conceptual models for time-dependent multimedia data

01 Aug 1993-IEEE Transactions on Knowledge and Data Engineering (IEEE Educational Activities Department)-Vol. 5, Iss: 4, pp 551-563
TL;DR: N-ary and reverse temporal relations are introduced and defined along with their temporal constraints to ensure a property of monotonically increasing playout deadlines to facilitate both real-time deadline-driven playout scheduling or optimistic interval-based process playout.
Abstract: Multimedia data often have time dependencies that must be satisfied at presentation time. To support a general-purpose multimedia information system, these timing relationships must be managed to provide utility to both the data presentation system and the multimedia author. New conceptual models for capturing these timing relationships, and managing them as part of a database are proposed. Specifically, n-ary and reverse temporal relations are introduced and defined along with their temporal constraints. These new relations are a generalization of earlier temporal models and establish the basis for conceptual database structures and temporal access control algorithms to facilitate forward, reverse, and partial-interval evaluation during multimedia object playout. The proposed relations are defined to ensure a property of monotonically increasing playout deadlines to facilitate both real-time deadline-driven playout scheduling or optimistic interval-based process playout. A translation of the conceptual models to a structure suitable for a relational database is presented. >

Summary (6 min read)

1 Introduction

  • Multimedia refers to the integration of text, images, audio, and video in a variety of application environments.
  • The task of coordinating sequences of these data requires synchronization among the interacting media as well as within each medium.
  • Consider the familiar canned multimedia slide presentation in which a series of verbal annotations coincides with a series of images.
  • Time-dependent multimedia objects require special considerations for presentation due to their real-time playout characteristics as data need to be delivered from the storage devices based on a prespecified schedule.
  • In Section 3, the authors describe interval-based modeling schemes, including their proposed models for establishing a conceptual database structure.

3 Interval-Based Conceptual Models

  • In order to support time-dependent data retrieval from a MDBMS, the authors must provide both conceptual temporal models for describing the data and models for their storage.
  • The authors introduce conceptual models that describe temporal information necessary to represent multimedia time dependencies and synchronization.
  • Specifically, this includes a discussion of the various temporal specification methodologies, their proposed reverse and n-ary temporal relations, and their partial-interval evaluation scheme.

3.1 Basic Temporal Relations

  • An important and often used representation of time is the temporal interval [1].
  • This functionality proved valuable for describing the temporal component of composite multimedia objects as shown by Little and Ghafoor [17].
  • For binary temporal relations, the duration, τTR, of two related intervals τα and τβ can be uniquely determined by the durations of intervals τα, τβ, τδ, and the temporal relation, tr, between them.
  • Like the binary temporal relations, there are thirteen possible n-ary temporal relations, which reduce to the seven cases indicated in Fig. 5, after eliminating their inverses.
  • The authors now investigate the properties of the n-ary temporal relations including implications of multiple playout deadlines and temporal constraints.

3.2.1 Property of Monotonic Playout Times

  • Therefore, the authors concentrate on the relations before, meets, overlaps, during−1, starts, finishes−1, and equals.
  • This does not affect their ability to model temporal relations, i.e., the authors have chosen a set of the 13 temporal relations such that an ordering relationship is always implied and is easily identified in both the forward and reverse directions.

3.2.2 Deadline Determination

  • A complex multimedia data object consists of many subobjects, each with characteristic time dependencies, and can be evaluated for the purpose of identifying the exact playout deadlines of each subobject.
  • This task is necessary for real-time scheduling of the retrieval of objects in the presence of significant system delays [18].
  • The following theorem describes the relative playout time for any object or start point of a temporal interval [19].
  • Theorem 1 specifies how to generate playout times from an n-ary temporal relation.
  • In Section 5.2 the authors show this theorem applied to an algorithm for identifying playout times from their proposed temporal model.

3.2.3 Temporal Constraints

  • Like the binary temporal relations described in Section 3.1, a set of constraints can be identified for the timing parameter relationships among intervals of the n-ary case.
  • 3 Fig. 6 illustrates these parameters for 3Similar to the binary cases, τTRn = max{τ 1, ∑n−1 i=1 τ i δ +τ n} for both the parallel and sequential relations, assuming τ idelta ≥ 0. the before and overlaps relations.
  • The τTRn are proven by induction using base cases from Table 1 as follows.
  • The notion of temporal intervals can also support reverse and partial playout activities, i.e., reversing the direction in time of presentation, or beginning the presentation of an object at a midpoint rather than at the beginning or end.
  • For this purpose, reverse temporal relations are proposed in the next section.

3.3 Reverse Temporal Relations

  • As mentioned earlier, in addition to simple linear playout of time-dependent data sequences, other modes of information access are also viable, and should be supported by a MDBMS.
  • These TAC operations include reverse playout and partial interval evaluation for midpoint suspension and resumption.
  • In this section the authors describe temporal models for satisfying these aforementioned requirements.
  • In order to facilitate reverse playout, the authors first characterize reversal of time in temporal intervals, and then show their n-ary extension.

3.3.1 Reverse Binary Temporal Relations

  • The following definitions and lemmas characterize reverse temporal intervals and relations: Definition 3 Let [a, b] and [c, d] be two temporal intervals related by tr, then the reverse temporal relation trr is defined by the temporal relation formed between [−b,−a] and [−d,−c].
  • The reverse relations are summarized in Fig. 8, noting that the reverse intervals are the reflection across a line on the time axis.
  • To identify reverse temporal parameters (τα−r, τβ−r, τδ−r, and trr) from the forward temporal parameters (τα, τβ, τδ, and tr), the conversions summarized in Table 3 can be used, which are derived from the consistency formulae of Table 1 and by inspection of the binary reverse relations of Fig.

3.3.3 Partial Interval Evaluation

  • This operation is typical during audio and video editing, in which a segment is repeatedly started, stopped, and restarted.
  • Another example of partial playout occurs when a viewer stops a motion picture then later restarts at some intermediate point (or perhaps an earlier point to get a recap).
  • In this section the authors show the basis for achieving partial evaluation or fractional playout for a composite (n-ary) temporal interval.
  • 5The same difficulty arises again for the during−1 relation.
  • A non-decomposable, or atomic, interval does not have an n-ary decomposition, and can represent the presentation of a data element.

4 Database Models for Time-Dependent Media

  • In Section 3 the authors have shown a new kind of temporal relation which can describe the timing among sets of multimedia objects.
  • The authors now show how the proposed relations and their associated formulation lead to the development of conceptual temporal models suitable for storage and retrieval of time-dependent multimedia data.
  • The authors discussion here parallels this earlier work, however, the new temporal relations provide a more efficient and homogeneous structure and provide new TAC functionality.
  • The authors do not assume any specific model for the multimedia database management component, rather, they build a framework which can be integrated, or used in a complementary fashion, with other DBMS schemes.
  • Later, the authors show examples of this conceptual model applied to a relational data model.

4.1 Proposed Temporal Data Model

  • To capture the semantics of the n-ary temporal relations and the object orchestration technique, the authors need to group multimedia objects and identify them with temporal parameters.
  • The first node type template for this purpose is the leaf or terminal node as indicated in Fig. 10(a).
  • This node has attributes which indicate media type (text, image, video, etc.) and a pointer which indicates the storage location of the data for presentation.
  • Note that the forward temporal parameters are necessary and sufficient for reconstituting a temporal relation and deriving the reverse parameters [17].
  • Because the authors have a set of slide-talk pairs, a single nonterminal node type is indicated to aggregate the set of sequentially related object pairs.

4.2 Database Creation

  • The construction of a multimedia database is the process of assigning data instances to the developed database model.
  • The authors assume multimedia data elements are stored and managed by some entity which can be accessed by pointers from terminal nodes defined by the conceptual model.
  • To establish temporal parameters for nonterminal nodes, Lemma 1 is used by combining timing parameters of temporally related terminal elements and assigning them to their connecting nonterminal parent nodes.
  • As this process is repeated, timing values are propagated up the database hierarchy until timing values are established at all nodes.
  • Determination of the reverse delay parameters can be performed using the conversion formulae provided in Section 3.

4.3 A Relational Data Model for Temporal Data

  • Using the conceptual model described above (Section 4.1), data structures can be implemented to capture time-dependent data.
  • The duration attribute describes the overall temporal interval for a possibly complex object.
  • Lower values of these indices indicate preceding (in time) subnodes, with increasing values indicating directional traverse of the temporal hierarchy for evaluation of the parent’s temporal relation.
  • This separate relation is defined apart from the nodes relation because many nodes are not terminal elements and do not require these attributes.
  • To map this conceptual model to the relational data structures the authors first identify each node with a unique identifier, and assign each terminal element to a tuple in the terminals relation along with its location and media type.

5 Access Algorithms for Multimedia Databases

  • The authors have presented a specification methodology and conceptual models suitable for database storage of time-dependent multimedia data.
  • In conjunction with these models, the authors have developed access algorithms that they describe in this section.
  • These algorithms provide functionality for partial interval playout, playout reversal, and playout deadline determination as required for real-time scheduling approaches.

5.1 Playout Scheduling Approaches

  • In formulating an approach to media delivery suitable for satisfying the timing requirements for multimedia data, one must realize that most workstation technology lacks sufficient performance to support the playout of complex multimedia objects.
  • For systems with marginal performance, techniques must be applied to guarantee delivery of data in a timely fashion.
  • These delays can be larger than the playout durations of individual data elements requiring synchronization, and a mechanism must be employed for compensation.
  • For both approaches the objects and their temporal hierarchy must be evaluated.
  • In the remainder of this section the authors show they show algorithms for TAC operations in the ideal case and an algorithm for generating deadlines for the real-time scheduled approach.

5.2 Determination of Playout Time Based on Temporal Intervals

  • The temporal hierarchy can be evaluated for the purpose of identifying the exact deadlines for playout of all objects by applying Theorem 1 for any object or start point of a temporal interval using either forward or reverse parameters.
  • Alternatively, the original OCPN model, used to describe a composite multimedia object, can be simulated directly [18].
  • The following algorithm, based on Theorem 1, finds the playout time, or deadline, for any object or start time of a temporal interval using either forward or reverse parameters.
  • This algorithm uses as input a parameter telapsed to indicate the current elapsed time throughout its processing, and is initially set to zero.
  • If Object is not a terminal then evaluate its subnodes: (a) For each subnode with increasing node index i: i.

5.3 Retrieval Algorithm for Reverse Playout

  • To support reverse playout of an object represented by their temporal data model the authors merely require evaluation of the reverse direction indices.
  • These can be derived from either existing forward temporal parameters or from the temporal relations themselves.
  • Rather than storing identical object durations, indices can be used to establish the playout ordering in the forward and reverse directions, hence the need for the position indices in the relational model.
  • The modification to the original presentation algorithm [17] to support forward and reverse playout is shown as a program fragment as follows (shown later as part of the complete playout algorithm): 1. For each subnode in increasing order of index id: where id is either the forward or reverse position index.
  • This functionality is provided though partial-interval evaluation.

5.4 Partial-Interval Evaluation

  • A further enhancement required for multimedia presentation is the ability to playout partial objects.
  • Partial interval evaluation allows us to stop in the middle of presentation and resume in either the forward or reverse direction.
  • For the fractional part, the authors must consider both nondecomposable and decomposable intervals.
  • For the reverse direction, the reverse elapsed playout time, offsetr, can be computed by subtracting the forward elapsed playout time from the overall duration of the object under consideration, i.e., offsetr = τobject − offsetf Based on Theorems 4 and 5, the authors describe the modifications necessary to support partial interval evaluation.
  • Recursively invoke algorithm on the ith subnode with offset.

5.5 Complete Playout Algorithm

  • The complete algorithm is shown below, considering all of the proposed modifications to their original presentation algorithm to support TAC functionality.
  • In this algorithm offset indicates either the forward or reverse elapsed time depending on the direction d of playout.
  • For object Object identify subnodes, and direction d. 2. If Object is not a terminal then evaluate its temporal relation: (a) If temporal relation is sequential then: i. For each subnode in increasing order of index id: A. If 0 ≤ offset < τ i then Recursively invoke algorithm on the idth subnode with offset.
  • If Object is a terminal then present data on the appropriate I/O device for its specified duration.
  • Terminate when all recursive invocations have completed.

6 Discussion

  • The proposed n-ary temporal relations offer a more elegant means of representing timing information than their previous binary models.
  • These formulae can be used at the time of object creation to establish a consistent database of temporal parameters by using a tree traversal procedure akin to the playout algorithms (not shown here).
  • The penalty for this computation depends on the complexity of the represented objects and their frequency of reversal.
  • Frequent sequence reversal suggests the use of stored reverse parameters whereas infrequent reversal can be adequately provided by dynamic parameter computation.
  • In this system the relational data structures supporting object timing complement an application-specific schema for a database of motion pictures.

7 Conclusion

  • In this paper the authors have proposed two new temporal models for time-dependent data retrieval in a multimedia database management system.
  • These models, the n-ary and reverse temporal relations, are shown to be useful for facilitating storage and retrieval of time-dependent multimedia data by applying a temporal hierarchy supported by a relational data model.
  • Because the proposed models are restricted to having the property of monotonically increasing playout deadlines for represented objects, algorithms for synchronous data retrieval are greatly simplified, both for identification of deadlines necessary for real-time scheduling of playout, or for optimistic process playout synchronization.
  • Furthermore, algorithms are shown that allow traversal of the temporal hierarchy in a manner that provides forward, reverse, or partial interval evaluation.

Did you find this useful? Give us your feedback

Content maybe subject to copyright    Report

Interval-Based Conceptual Models
for Time-Dependent Multimedia Data
1
T.D.C. Little
and A. Ghafoor
Multimedia Communications Laboratory
Department of Electrical, Computer and Systems Engineering
Boston University, Boston, Massachusetts 02215, USA
tdcl@bu.edu
School of Electrical Engineering
Purdue University, West Lafayette, Indiana 47907, USA
MCL Technical Repor t 05-07-1993
Abstract–Multimedia data often have time dependencies that must be satisfied at presen-
tation time. To support a general-purpo se multimedia information system, these timing
relationships must be managed to provide utility to both the data presentation system and
the multimedia author.
In this paper we propose new conceptual models for capturing these timing relationships
and managing them as part of a database. Specifically, we introduce and define n-ary
and reverse temporal relations along with their temporal constraints. These new relations
are a generalization of our earlier temporal models and establish the basis f or conceptual
database structures and temporal access control algorithms to facilitate forward, reverse, and
partial-interval evaluation during multimedia object playout. The proposed relations are
defined to ensure a property of monotonically increasing playout deadlines to facilitate both
real-time deadline-driven playout scheduling or optimistic interval-based process playout.
Furthermore, we show a translation of t he conceptual models to a structure suitable for a
relational database.
Keywords: Temporal Modeling, Multimedia Databases, Synchronization, Scheduling.
1
In IEEE Trans. on Knowledge and Data Engineering (Special Issue: Multimedia Information Systems),
Vol. 5, No. 4, August 1993, pp. 551- 563. Portions of this work were presented at the 199 2 Workshop on
Multimedia Informations Systems held in Tempe, Arizona on February 7, 1992. The work by T.D.C. Little
is supported in part by the National Science Foundation under Grant No. IRI-9211165. The wo rk by A.
Ghafoor is supported in part by the National Science Foundation under Grant No. CDA-912177 1. The
authors also acknowledge the support of the New York State Center for Advanced Technology in Computer
Applications and Software Engineering (CASE) at Syracuse University.

1 Introduction
Multimedia refers to the integration of text, images, audio, and video in a variety of appli-
cation environments. These data can be heavily time-dependent, such as audio and video
in a motion picture, and require time-ordered playout during presentation. The task of co-
ordinating sequences of these data r equires synchronization among the interacting media as
well as within each medium. Synchronization can be applied to the playout of concurrent
or sequential streams of data and to the external events generated by a human user includ-
ing browsing, querying, and editing typical of stored-data applications. This problem of
synchronizing time-ordered sequences of data element s is fundamental to multimedia data.
Timing relationships between the media can be implied, as in the simultaneous acqui-
sition of voice and video, or can be explicitly formulated, as in the case of a multimedia
document with voice-annotated text. In either situation, the characteristics of each medium,
and the relationships among them must be established in order to provide synchronization
in the presence of vastly different presentation requirements. Consider the familiar canned
multimedia slide presentation in which a series of verbal annotations coincides with a series
of images. The presentation of the annotations and the slides occur in a sequential man-
ner. Synchronization points correspond to the change o f an image and the end of a verbal
annotation, representing a coarse-gra in synchronization between objects. A fine-g r ained ex-
ample of synchronization is the lip-sync of audio and video which usually requires 25 or 30
synchronization points per second.
A multimedia system must preserve the timing relationships among the elements of the
object presentation at these points by the process of multimedia synchro nization. A mul-
timedia database management system (MDBMS) must have the capability for managing
the aspects of time required for time-dep endent media. This problem is different from t he
provision o f historical databases, temporal query languages [25, 27], or time-critical query
evaluation [14]. Time-dependent multimedia objects require special considerations for pre-
sentation due to their real-time playout characteristics as data need to be delivered from
the storage devices based on a prespecified schedule. Furthermore, presentation of a single
object can occur over an extended duration (e.g., a motion picture). Fig. 1 illustrates such
a time dependency between elements of a compo site multimedia object. In this example, a
sequence of text and image elements are played-out in succession based on such a prespecified
schedule.
The tolerance of data to timing skew and jitter during playout varies widely depending on
2

time
1
t
2
t
3
t
4
t
5
t
6
t
7
t
Figure 1: Time-Dependent Data Presentation
the medium. Audio and video require tight b ounds on the order of hundreds of milliseconds,
whereas synchronous text and images can tolerate skew on the order of seconds. Furthermore,
audio and video can tolerate different absolute timing requirements during playout as the
human ear can discern dropouts in audio data more readily than of video. Based on the
data’s tolerance to skew a nd jitter during playout, two approaches to providing synchronous
playout of time-dependent data streams have been propo sed. These consist of a real-time
scheduling approach [19], and an optimistic interval-based process approach as proposed in
this paper.
In addition to simple linear playout of time-dependent data sequences, other modes of
data access are also possible due to the unique nature of the multimedia data objects and
should be supported by a MDBMS. These include,
Reverse
Fast-forward
Fast-backward
Midpo int suspension
Midpo int resumption
These Temporal Access Control (TAC) operations are feasible with existing technologies;
however, when non-sequential stora ge devices are used with complex data compression al-
gorithms, and random communication delays are introduced due to data distribution, the
provision of these capabilities can be very difficult. Examples include viewing a motion
picture backwards or reversing an animation of a series o f images (sequence reversal), rapid
viewing of a long sequence o f time-dependent data by increasing the rate of presentation
or by skipping some data (fast-forward or fast-backward), and stopping and starting of a
3

motion picture at an arbitrary point (midpoint suspension, resumption and partial interval
evaluation).
In this paper, we propose temporal-interval-based models and constraints which provide
a basis for a proposed conceptual data representation and algorithmic support of the afo re-
mentioned TAC functionality. The work represents major extension and generalization o f
our earlier models presented in [17]; however, we do not consider the dynamic properties
of user interaction (e.g., Stotts and Furuta [24]). The uncertainty created by random user
interaction is an a dditional complexity in managing time in multimedia information systems.
The remainder of the paper is organized as f ollows. In Section 2, we review related work
on time-dependent data storag e. In Section 3, we describe interval-based modeling schemes,
including our proposed models for establishing a conceptual database structure. Section 4
describes a conceptual data representation based on the new temporal models, including an
example using a relational implementation. Section 5 describes algo rithms for accessing the
proposed conceptual models in the context of a database. We discuss characteristics of the
overall modeling methodology in Section 6, and in Section 7 we conclude the paper.
2 Background and Related Work
The primary requirements for the support of time-dependent data playout in an MDBMS
include the means for the identification of temporal relations between multimedia da ta ob-
jects, temporal conceptual database schema development, physical schema design, and syn-
chronous a ccess for data retrieval. In this section we briefly describe these requirements,
introduce various terminology, and describe related work.
As indicated in the introduction, time-dependent data differ from historical data which
do not specifically require timely playout. Typically, time-dependent data are stored using
mature technologies possessing mechanisms to ensure synchronous playout (e.g., VCRs or
audio tape recorders). With such mechanisms, dedicated hardware provides a constant rate
of playout for homogeneous, periodic sequences of data, and concurrency in data streams
is provided by independent physical data paths. When this type of data is migrated to
more general-purpose computer data storage systems (e.g., disks), many interesting new
capabilities are possible, including random access to the tempora l dat a sequence and time-
dependent playout of static data (animation). However, the generality of such a system
eliminates the dedicated physical data paths and the implied data structures of sequentia l
4

storage. Therefore, a general MDBMS needs to support new access para digms including a
retrieval mechanism for large amounts of multimedia data, and must provide conceptual and
physical database schemata to support these paradigms. Furthermore, a MDBMS must also
accommodate the performance limitations of the computer.
At the conceptual level, the temporal aspects of data must be modeled. Da t a can have
natural or implied time dependencies, (e.g., audio and video r ecorded simultaneously). These
data streams often are described as continuous because recorded data elements form a con-
tinuum during playout, i.e., elements are played-out contiguously in time. Static data,
which lack time dependencies, can have synthetic temporal relationships (e.g., Fig. 1). The
combination of natural and synthetic time dependencies can describe the overall tempor al
requirements of any pre-orchestrated multimedia presentation. Temporal information can be
encapsulated in the description of the data using the object-oriented paradigm (e.g., Gibbs
[10], Herrtwich [12]). Using such schemes, tempor al information such as a time reference,
playout time units, tempo ral relationships, and required time offsets can be maintained for
specific multimedia objects. If the data are periodic, this approach can define the time
dependencies for an entire sequence by defining the period or frequency of playout (e.g.,
30 frames/s for video). For mixed-type, time-dependent data, there have been several pro-
posals for their conceptual modeling and interchange format specification, most based on
temporal-interva l-based schemes [5, 8, 13, 23]. However, these works either neglect to con-
sider the implications on t he development of conceptual database structures to support TAC
operations or do not comprehensively model time-dependent data.
Once a conceptual temporal model is established for a multimedia object, the multimedia
data must be mapped to the physical system to facilitate database access and retrieval. For
time-dependent multimedia data this presents some interesting challenges. Problems arise
due to the strict timing requirements for playout of time-dependent data. The multimedia
types of audio and video require very large amounts of storage space and must be maintained
in secondary storage. In order to meet the presentation requirements for these data, vari-
ous physical storage organizations have been proposed, such as storing data in contiguous
blocks on a disk in the same order as playout. Some recent work on data placement on
physical storage for audio data retrieval is described by Yu et al. [28, 30], Gemmell and
Christo doulakis [9], Rangan and Vin [22], and Christodoulakis and Faloutsos [7]. We do not
address physical storage organizations here.
The integration of conceptual and physical data models with system supp ort for data
delivery yields the functionality necessary to construct multimedia applications. System
5

Citations
More filters
Journal ArticleDOI
TL;DR: This paper surveys recent studies on content-based retrieval for multimedia databases from the point of view of three fundamental issues: an underlying data model, a priori knowledge of the area of interest, and the scheme for representing queries.
Abstract: Conventional database systems are designed for managing textual and numerical data, and retrieving such data is often based on simple comparisons of text/numerical values. However, this simple method of retrieval is no longer adequate for multimedia data, since the digitized representation of images, video, or data itself does not convey the reality of these media items. In addition, composite data consisting of heterogeneous types of data also associates with the semantic content acquired by a user's recognition. Therefore, content-based retrieval for multimedia data is realized taking such intrinsic features of multimedia data into account. Implementation of the content-based retrieval facility is not based on a single fundamental, but is closely related to an underlying data model, a priori knowledge of the area of interest, and the scheme for representing queries. This paper surveys recent studies on content-based retrieval for multimedia databases from the point of view of three fundamental issues. Throughout the discussion, we assume databases that manage only nontextual/numerical data, such as image or video, are also in the category of multimedia databases.

388 citations

Patent
02 Mar 2002
TL;DR: In this article, a push-pull model for efficient low-latency video-content distribution over a network is proposed, where metadata is used as a vehicle and mechanism to enable intelligent decisions to be made on content distribution system operation.
Abstract: Method, system, computer program and computer program product for a metadata enabled push-pull model and method for efficient low-latency video-content distribution over a network. Metadata is used as a vehicle and mechanism to enable intelligent decisions to be made on content distribution system operation. Metadata is data that contains information about the actual content, and in some cases, the metadata may also contain portions of the content or a low-resolution preview of the content. Aspects of the invention are directed toward the distribution of metadata throughout the network in a way that facilitates efficient system operation as well as optionally but advantageously providing set of services such as tracking, reporting, personalization, and the like.

315 citations

Patent
03 Apr 1998
TL;DR: In this paper, a technique for representing a visual scene as a directed acyclic graph of data and operators that generates a sequence of image frames over specified time intervals is presented.
Abstract: A technique for representing a visual scene as a directed acyclic graph of data and operators that generates a sequence of image frames over specified time intervals. The graph specifies temporal and spatial values for associated visual elements of the scene. Time is modeled in the inheritance properties explicitly defined within the scene graph hierarchy, by assigning temporal attributes to each media element. Branch nodes of the graph specify transforms for the temporal and spatial coordinate systems. To evaluate the appearance or behavior of the scene and in particular the global time values of particular elements at a given time instant, the graph is traversed in a direction from a root node down toward the leaf nodes, thereby causing temporal transformations specified along the branches of the graph to modify time parameters of the scene data at the nodes. Child nodes are preferably evaluated after being transformed, to determine the extent to which they contribute the data to the final scene. Temporal transformations may include translation operations that offset temporal event times; scaling operations that change the rate at which time passes; or clipping operations, that restrict the range of time parameters to exclude the evaluation of parts of the graph.

288 citations

Proceedings ArticleDOI
01 Sep 1993
TL;DR: In this paper the general requirements of video-on-demand are described and a system supporting content-based retrieval and playback for the structure and content of digital motion pictures is introduced.
Abstract: Video-on-demand represents a key demonstrative application for enabling multimedia technology in communication, database, and interface research. This application requires solving a number of diverse technical problems including the data synchronization problem for time-dependent data delivery. In this paper we describe the general requirements of video-on-demand and introduce a system supporting content-based retrieval and playback for the structure and content of digital motion pictures. In our model we capture domain-specific information for motion pictures and provide access to individual scenes of movies through queries on a temporal database. We describe our implementation of this service using existing workstation and storage technology.

217 citations

Journal ArticleDOI
TL;DR: A knowledge-based approach to retrieve medical images by feature and content with spatial and temporal constructs is developed and the KMeD (Knowledge-based Medical Database) system is implemented using these concepts.
Abstract: A knowledge-based approach to retrieve medical images by feature and content with spatial and temporal constructs is developed. Selected objects of interest in an image are segmented and contours are generated. Features and content are extracted and stored in a database. Knowledge about image features can be expressed as a type abstraction hierarchy (TAH), the high-level nodes of which represent the most general concepts. Traversing TAH nodes allows approximate matching by feature and content if an exact match is not available. TAHs can be generated automatically by clustering algorithms based on feature values in the databases and hence are scalable to large collections of image features. Since TAHs are generated based on user classes and applications, they are context- and user-sensitive. A knowledge-based semantic image model is proposed to represent the various aspects of an image object's characteristics. The model provides a mechanism for accessing and processing spatial, evolutionary and temporal queries. A knowledge-based spatial temporal query language (KSTL) has been developed that extends ODMG's OQL and supports approximate matching of features and content, conceptual terms and temporal logic predicates. Further, a visual query language has been developed that accepts point-click-and-drag visual iconic input on the screen that is then translated into KSTL. User models are introduced to provide default parameter values for specifying query conditions. We have implemented the KMeD (Knowledge-based Medical Database) system using these concepts.

180 citations

References
More filters
Journal Article
TL;DR: An interval-based temporal logic is introduced, together with a computationally effective reasoning algorithm based on constraint propagation, which is notable in offering a delicate balance between space and time.
Abstract: An interval-based temporal logic is introduced, together with a computationally effective reasoning algorithm based on constraint propagation. This system is notable in offering a delicate balance between

7,466 citations

Journal ArticleDOI
TL;DR: In this paper, an interval-based temporal logic is introduced, together with a computationally effective reasoning algorithm based on constraint propagation, which is notable in offering a delicate balance between time and space.
Abstract: An interval-based temporal logic is introduced, together with a computationally effective reasoning algorithm based on constraint propagation. This system is notable in offering a delicate balance between

7,362 citations

Journal ArticleDOI
TL;DR: A technique is presented for the formal specification and modeling of multimedia composition with respect to intermedia timing and the synchronization requirements of complex structures of temporally related objects can be easily specified.
Abstract: A technique is presented for the formal specification and modeling of multimedia composition with respect to intermedia timing. The proposed model is based on the logic of temporal intervals and timed Petri nets. A strategy is evinced for constructing a database schema to facilitate data storage and retrieval of media elements based on the temporal relationship established by the proposed modeling tool. An algorithm which allows the retrieval of media elements from the constructed database in a manner which preserves the temporal requirements of the initial specification is presented. Using the proposed model, the synchronization requirements of complex structures of temporally related objects can be easily specified. >

616 citations

Proceedings ArticleDOI
02 Apr 1984
TL;DR: TQuel as discussed by the authors is a superset of Quel, the query language in the Ingres relational database management system, which is used to query historical databases (HDBs) representing an enterprise over time.
Abstract: Recently, attention has been focussed on historical databases (HDBs), representing an enterprise over time. We have developed a new language, TQuel, to query an HDB. TQuel is a superset of Quel, the query language in the Ingres relational database management system. This paper provides an overview of the language, motivating the various design decisions with the objective that it be a minimal extension, both syntactically and semantically, of Quel.

479 citations

Proceedings ArticleDOI
01 Sep 1991
TL;DR: A model that relates disk and device characteristics to the recording rate, and derive storage granularity and scattering parameters that guarantee continuous access is presented, which serves as a testbed for experimenting with policies and algorithms for multimedia storage.
Abstract: We address the unique requirements of a multimedia file system such as continuous storage and retrieval of media, maintenance of synchronization between multiple media streams, and efficient manipulation of huge media objects. We present a model that relates disk and device characteristics to the recording rate, and derive storage granularity and scattering parameters that guarantee continuous access. In order for the file system to support multiple concurrent requests, we develop admission control algorithms for determining whether a new request can be accepted without violating the realtime constraints of any of the requests.We define a strand as an immutable sequence of continuously recorded media samples, and then present a multimedia rope abstraction which is a collection of individual media strands tied together by synchronization information. We devise operations for efficient manipulation of multi-stranded ropes, and develop an algorithm for maintaining the scattering parameter during editing so as to guarantee continuous playback of edited ropes.We have implemented a prototype multimedia file system, which serves as a testbed for experimenting with policies and algorithms for multimedia storage. We present our initial experiences with using the file system.

330 citations

Frequently Asked Questions (1)
Q1. What contributions have the authors mentioned in the paper "Interval-based conceptual models for time-dependent multimedia data" ?

To support a general-purpose multimedia information system, these timing relationships must be managed to provide utility to both the data presentation system and the multimedia author. In this paper the authors propose new conceptual models for capturing these timing relationships and managing them as part of a database. Specifically, the authors introduce and define n-ary and reverse temporal relations along with their temporal constraints. Furthermore, the authors show a translation of the conceptual models to a structure suitable for a relational database.