BenchCouncil Transactions on Benchmarks, Standards and Evaluations (TBench) is an open-access, multi-disciplinary journal dedicated to advancing the science and engineering of evaluation. We extend a warm invitation to researchers from diverse disciplines to submit their work, with a particular emphasis on interdisciplinary research. Whether the research pertains to computers, artificial intelligence (AI), medicine, education, finance, business, psychology, or other social disciplines, all relevant contributions are highly valued and welcomed.At TBench, we place great importance on the reproducibility of research. We strongly encourage authors to ensure that their articles are appropriately prepared for open-source or artifact evaluation prior to submission. Areas of interest include, but are not limited to:1. Evaluation theory and methodologyFormal specification of evaluation requirementsDevelopment of evaluation modelsDesign and implementation of evaluation systemsAnalysis of evaluation riskCost modeling for evaluationsAccuracy modeling for evaluationsEvaluation traceabilityIdentification and establishment of evaluation conditionsEquivalent evaluation conditionsDesign of experimentsStatistical analysis techniques for evaluationsMethodologies and techniques for eliminating confounding factors in evaluationsAnalytical modeling techniques and validation of modelsSimulation and emulation-based modeling techniques and validation of modelsDevelopment of methodologies, metrics, abstractions, and algorithms specifically tailored for evaluations2. The engineering of evaluation Benchmark design and implementationBenchmark traceability Establishing least equivalent evaluation conditionsIndex design, implementationScale design, implementationEvaluation standard design and implementationsEvaluation and benchmark practice Tools for evaluationsReal-world evaluation systemsTestbed3. Data setExplicit or implicit problem definition deduced from the data setDetailed descriptions of research or industry datasets, including the methods used to collect the data and technical analyses supporting the quality of the measurementsAnalyses or meta-analyses of existing dataSystems, technologies, and techniques that advance data sharing and reuse to support reproducible researchTools that generate large-scale data while preserving their original characteristicsEvaluating the rigor and quality of the experiments used to generate the data and the completeness of the data description4. BenchmarkingSummary and review of state-of-the-art and state-of-the-practiceSearching and summarizing industry best practiceEvaluation and optimization of industry practiceRetrospective of industry practiceCharacterizing and optimizing real-world applications and systemsEvaluations of state-of-the-art solutions in the real-world setting5. Measurement and testingWorkload characterizationInstrumentation, sampling, tracing, and profiling of large-scale, real-world applications and systemsCollection and analysis of measurement and testing data that yield new insightsMeasurement and testing-based modeling (e.g., workloads, scaling behavior, and assessment of performance bottlenecks)Methods and tools to monitor and visualize measurement and testing dataSystems and algorithms that build on measurement and testing-based findingsReappraisal of previous empirical measurements and measurement-based conclusionsReappraisal of previous empirical testing and testing-based conclusionsEditorial Board
The journal aims to promote and communicate advances in big data research by providing a fast and high quality forum for researchers, practitioners and policy makers from the very many different communities working on, and with, this topic.The journal will accept papers on foundational aspects in dealing with big data, as well as papers on specific Platforms and Technologies used to deal with big data. To promote Data Science and interdisciplinary collaboration between fields, and to showcase the benefits of data driven research, papers demonstrating applications of big data in domains as diverse as Geoscience, Social Web, Finance, e-Commerce, Health Care, Environment and Climate, Physics and Astronomy, Chemistry, life sciences and drug discovery, digital libraries and scientific publications, security and government will also be considered. Occasionally the journal may publish whitepapers on policies, standards and best practices.Benefits to authorsWe also provide many author benefits, such as free PDFs, a liberal copyright policy, special discounts on Elsevier publications and much more. Please click here for more information on our author services .Please see our Guide for Authors for information on article submission. If you require any further information or help, please visit our Support Center.
Informatics in Medicine Unlocked (IMU) is an international gold open access journal covering a broad spectrum of topics within medical informatics, including (but not limited to) papers focusing on imaging, pathology, teledermatology, public health, ophthalmological, nursing and translational medicine informatics. The full papers that are published in the journal are accessible to all who visit the website.Manuscripts exploring the revolutionary advances being made in the application of the computer to the fields of bioscience and medicine can be submitted to the journal?s companion title, Computers in Biology and Medicine .Only manuscripts of sufficient quality with scientific relevance will be considered for publication in IMU; the types of articles accepted include original research papers, reviews, perspectives, letters to the editor and editorials, tutorials, and papers on innovative and topical new technology. Proposals for special focus issues from leading research groups will also be considered.Manuscript referees will be experts in the subject matter, working in academia, industry, and at government institutions. Usually, comments from 2-5 referees will be returned to the authors after first submission of the manuscript. Average turnaround time for manuscript evaluation is expected to be less than 4 weeks. Average time from first submission to publication of accepted papers, including revision, will typically be 2-4 months.
Databases: Their Creation, Management and UtilizationInformation systems are the software and hardware systems that support data-intensive applications. The journal Information Systems publishes articles concerning the design and implementation of languages, data models, process models, algorithms, software and hardware for information systems.Subject areas include data management issues as presented in the principal international database conferences (e.g., ACM SIGMOD/PODS, VLDB, ICDE and ICDT/EDBT) as well as data-related issues from the fields of data mining/machine learning, information retrieval coordinated with structured data, internet and cloud data management, business process management, web semantics, visual and audio information systems, scientific computing, and data science. We welcome systems papers that focus on implementation considerations in massively parallel data management, fault tolerance, and special purpose hardware for data-intensive systems; theoretical papers that either break significant new ground or unify and extend existing algorithms for data-intensive applications; and manuscripts from application domains, such as urban informatics, social and natural science, and Internet of Things, which present innovative, high-performance, and scalable solutions to data management problems for those domains.All papers should motivate the problems they address with compelling examples from real or potential applications. Systems papers must be serious about experimentation either on real systems or simulations based on traces from real systems. Papers from industrial organizations are welcome. Theoretical papers should have a clear motivation from applications and clearly state which ideas have potentially wide applicability.Authors of selected articles that have been accepted for publication in Information Systems are invited by the EiCs to submit the experiment described in their papers for reproducibility validation. The resulting additional reproducibility paper is co-authored by the reproducibility reviewers and the authors of the original publication.As part of its commitment to reproducible science, Information Systems also welcomes experimental reproducible survey papers. Such submissions must: (i) apply a substantial portion of the different surveyed techniques to at least one existing benchmark and perhaps one or more new benchmarks, and (ii) be reproducible (the validation of reproducibility will result in a separate paper following the guidelines of our Reproducibility Editor).In addition to publishing submitted articles, the Editors-in-Chief will invite retrospective articles that describe significant projects by the principal architects of those projects. Authors of such articles should write in the first person, tracing the social as well as technical history of their projects, describing the evolution of ideas, mistakes made, and reality tests. We will make every effort to allow authors the right to republish papers appearing in Information Systems in their own books and monographs.
For JSS's full CfP including information on Special Issues, Industry, Trends, and Journal First tracks please continue to read for further details.The Journal of Systems and Software publishes papers covering all aspects of software engineering. All articles should provide evidence to support their claims, e.g. through empirical studies, simulation, formal proofs or other types of validation. Topics of interest include, but are not limited to: Methods and tools for software requirements, design, architecture, verification and validation, testing, maintenance and evolutionAgile, model-driven, service-oriented, open source and global software developmentHuman/social aspects in software engineering and developerexperienceArtificial Intelligence, data analytics and big data applied in software engineeringMetrics and evaluation of software development resourcesDevOps, continuous integration, build and test automationSoftware Engineering educationEthical/societal aspects of Software EngineeringSoftware Engineering for AI systemsSoftware Engineering for SustainabilityMethods and tools for empirical software engineering research The journal welcomes reports of practical experience for all of these topics, as well as replication studies and studies with negative results. The journal appreciates the submission of systematic literature reviews, mapping studies and meta-analyses. However, these should report interesting and important results, rather than merely providing statistics on publication year, venue etc.JSS supports Open Science and reproducible research. Therefore, authors are encouraged to make Open Science material available at the time of submission and after acceptance of their manuscript, e.g., by submitting artifacts related to a study to an archived open repository (such as arXiv.org, zenodo.org, Mendeley, etc.). Also, authors are encouraged to explicitly reference Open Science material in their manuscript (e.g., via a DOI from the open repository). If authors are not able to disclose any material (for example, industrial data subject to non-disclosure agreements), we encourage authors to explicitly acknowledge this by including a short statement in their manuscript. Depending on the type of research presented in a manuscript, Open Science material could include study protocols, (anonymized) raw or analyzed data, data analysis scripts, source code, customized tools and infrastructures, experimental material, codebooks, etc. If authors agree to participate in the JSS Open Science Initiative, after the acceptance of a manuscript, they will be invited to submit a link to Open Science material for review by the JSS Open Science Board. After a successful review (which does not impact the acceptance of the manuscript) considering availability and usability of the material, the publisher will add a statement to the final version of the manuscript acknowledging that the Open Science package was validated by the JSS Open Science Board.In addition to regular papers, JSS features two special tracks (In Practice, New Ideas and Trends Papers), as well as special issues.In Practice is exclusively focused on work that increases knowledge transfer from industry to research. It accepts: (1) Applied Research Reports where we invite submissions that report results (positive or negative) concerning the experience of applying/evaluating systems and software technologies (methods, techniques and tools) in real industrial settings. These comprise empirical studies conducted in industry (e.g., action research, case studies) or experience reports that may help understanding situations in which technologies really work and their impact. Submissions should include information on the industrial setting, provide motivation, explain the events leading to the outcomes, including the challenges faced, summarize the outcomes, and conclude with lessons learned, take-away messages, and practical advice based on the described experience. Contributing authors from industry are encouraged but not mandatory. (2) Practitioner Insights where we invite experience reports showing what actually happens in practical settings, illustrating the challenges (and pain) that practitioners face, and presenting lessons learned. Problem descriptions with significant details on the context, underlying causes and symptoms, and technical and organizational impact are also welcome. Practitioner insights papers may also comprise invited opinionated views on the evolution of chosen topic areas in practice. In contrast to applied research reports, practitioner insights are limited to four pages and the first author must be from industry. Finally, submissions to this track should be within scope of the journal's above topics of interest and they will be evaluated through industry-appropriate criteria for their merit in reporting useful industrial experience rather than in terms of academic novelty of research results.New Ideas and Trends Papers New ideas, especially those related to new research trends, emerge quickly. To accommodate timely dissemination thereof, JSS introduces the New Ideas and Trends Paper (NITP). NITPs should focus on the systems/software engineering aspects of new emerging areas, including: the internet of things, big data, cloud computing, software ecosystems, cyber-physical systems, green/sustainable systems, continuous software engineering, crowdsourcing, and the like. We distinguish two types of NITPs:A short paper that discusses a single contribution to a specific new trend or a new idea.A long paper that provides a survey of a specific trend, as well as a (possibly speculative) outline of a solution.NITPs are not required to be fully validated, but preliminary results that endorse the merit of the proposed ideas are welcomed.We anticipate revisiting specific new trends periodically, for instance through reflection or progress reports. New Ideas and Trend Papers warrant speedy publication.Special Issue proposals To submit a proposal for a special issue please submit your proposal here to Special Issues Editors Prof. Raffaela Mirandola and Prof. Laurence Duchien. Please visit the special issue guidelines page first to review the proposal guidelines and to download the proposal template required when submitting a proposal.Journal First Initiative Authors of JSS accepted papers have the opportunity to present their work in those conferences that offer a Journal First track. Using this track, researchers may take the best from two worlds: ensuring high quality in the JSS publication (thorough, multi-phase review process of a long manuscript), while getting feedback from a community of experts and fostering possible collaborations during a scientific event.Details may vary from conference to conference, but generally speaking, JSS papers to be presented in a Journal First track must report completely new research results or present novel contributions that significantly extend previous work. The ultimate decision to include a paper in the conference program is up to the conference chairs, not JSS. A JSS paper may be presented only once through a Journal First track.As of today, the list of conferences with which JSS is collaborating, or has collaborated, through a Journal First track, is: ASE, ICSME, SANER, RE, ESEM, PROFES, and APSEC.
The vast accumulation of health-related and biomedical data resources and the rapid proliferation of technological developments in data analytics are opening up new avenues to gaining insight in complex biological processes. High-throughput and precision measurement technologies are generating at a rapid pace large multimodal data sets that are distributed globally. However, new methods and infrastructures to extract, pool, integrate, refine, store, secure, analyse and visualize data are needed to unleash the power of these data resources, while tools and workflows should be made more accessible and easier to use by researchers and the public at large.Management and stewardship of life science and health data resources and subsequent analytics come with distinct domain challenges: Increasingly, a complex molecular focus requires joint expertise in the life science and health domain and data science domain, often requiring translation between the two in order to foster technology push and pull. Molecular science is done in ever larger domain consortia, where the emergence of community embraced standards is essential.The journal invites data science contributions covering foundational and theoretical research, platforms, infrastructure, methods, applications, and tools in molecular life sciences and biomedicine. We also welcome contributions aimed at fostering community engagement and agile best-practice development. Various submission types are facilitated (research/use case/infrastructure long articles; white paper/best practices/education short articles). As the remit of the journal is molecular life sciences and health, manuscripts must revolve around a central biological question.Data Science challenges include:ReproducibilityExperimental designData analyticsScalabilityPrivacyCommunities include those inMetabolic diseasesCancerInfectious diseasesNeurodegenerative diseasesNutritionTopics within the scope of the journal include (but are not limited to) the design, development, evaluation or validation of the following data science technology aspects in molecular life science and health:DATA SCIENCE TECHNOLOGY Data management and stewardshipData curationFAIR data principlesData integrationResearch data publication, quality, indexing and discoveryInfrastructure developmentPrivacy-aware analyticsMachine learning, deep learningNatural language processing and text-miningSemanticsTrend discovery and analysisGraph mining and knowledge extractionSocial and wearable sensorsScientific web services and executable workflowsMOLECULAR LIFE SCIENCE AND HEALTH REMIT Integration of omics, biomedical, nutrition, lifestyle and/or social data and subsequent analyticsMulti-level integration of molecular processesPersonalized and precision medicineDisease diagnosis, prognosis and prognostics