University of Paderborn

About: University of Paderborn is a education organization based out in Paderborn, Nordrhein-Westfalen, Germany. It is known for research contribution in the topics: Control reconfiguration & Software. The organization has 6684 authors who have published 16929 publications receiving 323154 citations.

AdapterDrop: On the Efficiency of Adapters in Transformers

Abstract: Massively pre-trained transformer models are computationally expensive to fine-tune, slow for inference, and have large storage requirements. Recent approaches tackle these shortcomings by training smaller models, dynamically reducing the model size, and by training light-weight adapters. In this paper, we propose AdapterDrop, removing adapters from lower transformer layers during training and inference, which incorporates concepts from all three directions. We show that AdapterDrop can dynamically reduce the computational overhead when performing inference over multiple tasks simultaneously, with minimal decrease in task performances. We further prune adapters from AdapterFusion, which improves the inference efficiency while maintaining the task performances entirely.

Event-Triggered Quantized Communication-Based Distributed Convex Optimization

Abstract: A NOVEL distributed algorithm based on multiple agents with continuous-time dynamics is proposed for a convex optimization problem where the objective function is the summation of local objective functions and the state of each agent is subject to a convex constraint set. Considering the limited bandwidth of the communication channels, we introduce a dynamic quantizer for each agent. To further save on communication costs, we develop an event-based broadcasting scheme for each agent. In comparison with algorithms that rely on continuous communication, the proposed algorithm serves to save communication expenditure by exploiting temporal and spatial aspects. Though a joint design of dynamic quantizers and event-trigger functions are under mild conditions, the states of the agents asymptotically approach the global optimal point with an adjustable error bound without incurring Zeno behavior.

Digital simulation of nonlinear circuits by wave digital filter principles

Abstract: The known principles of wave digital filters are used to simulate certain nonlinear circuits by digital means. Nonlinear elements-in particular, resistances having a continuous piecewise-linear voltage-current characteristic-are considered. It turns out that a representation of a nonlinear resistance in terms of the wave variables may exist, even if the voltage-current characteristic is neither voltage-controlled nor current-controlled. As an example, Chua's circuit is considered. It is shown that in the wave digital model of this circuit, essentially the same double-scroll attractor can be observed as in the corresponding analog realization. >

Load balancing in large networks: a comparative study

Abstract: The authors compare six well known and two new load balancing strategies on torus and ring topologies of different sizes and workload characteristics. Through simulations on a large transputer network, they show that all strategies behave differently under the workload of process and data migration. The two new algorithms based on the gradient model method are shown to be robust to both kinds of workloads. Thus, these new algorithms are good candidates for distributed operating systems running on large networks, where the workload characteristics can not be determined in advance. >

An Alternative to Mapping a Word onto a Concept in Language Acquisition: Pragmatic Frames.

Abstract: The classic mapping metaphor posits that children learn a word by mapping it onto a concept of an object or event. However, we believe that a mapping metaphor cannot account for word learning, because even though children focus attention on objects, they do not necessarily remember the connection between the word and the referent unless it is framed pragmatically, that is, within a task. Our theoretical paper proposes an alternative mechanism for word learning. Our main premise is that word learning occurs as children accomplish a goal in cooperation with a partner. We follow Bruner's (1983) idea and further specify pragmatic frames as the learning units that drive language acquisition and cognitive development. These units consist of a sequence of actions and verbal behaviors that are co-constructed with a partner to achieve a joint goal. We elaborate on this alternative, offer some initial parametrizations of the concept, and embed it in current language learning approaches.