Each submitted paper will be reviewed by at least three experts in the field. Submissions will be judged on the following criteria.
- Suitability: Does the contribution fit to the scope of QoMEX and in particular QoE, UX, immersive experiences, as well as to the list of topics of interest?
- Originality: What new ideas, approaches and/or results are introduced?
- Significance/contribution: Is the paper relevant for QoE and UX research? What is the benefit that others can gain from the contribution?
- Validity of the work: How confidently can researchers and practitioners use the results? Is the paper technically/methodologically sound? E.g. measurement setup, design of studies, conduction of simulation studies, statistical analysis, system modeling.
- Related work: Is state-of-the-art relevant to the described work sufficiently addressed? Does the paper refer to peer-reviewed publications?
- Thoroughness: Is the research carried out in a thorough way? Are the discussions and conclusions detailed enough?
- Replicability: Are the results and conclusions clearly described and reproducible? Are public data sets or open source code available?
- Presentation clarity: Is the structure of the text and the presentation of the results clear?
Submitted papers of the general and special session tracks will undergo a double-blind review process. The dataset & tools track as well as the demo track will follow a single-blind review process as authors need to include a link to the dataset, tool, or demo which will be evaluated by the reviewers.
Each paper is assigned a Meta-Reviewer who is responsible for coordinating the reviews of the paper and helping the TPC chairs in the decision process. The Meta-Reviewer is selected by the TPC chairs, and part of the program committee. The responsibility of the Meta-Reviewer, assigned 3-4 full papers, is as follows:
- Double check the reviews, ensuring high-quality standards.
- Write a meta-review, summarizing the strengths and weaknesses.
- Encourage discussion between the reviewers, update the meta-review if needed and make a recommendation to the TPC chairs about the paper.
- Participate in the (hybrid) TPC meeting (March 20, 2024).
There will be a paper bidding phase for short and full papers to ensure a good match between an individual reviewer’s expertise/interests and her/his review assignments. An online TPC meeting for full papers will take place prior to full paper notifications. Details will be provided later. All TPC Members are invited to participate in the TPC meeting.
For short papers, an online TPC meeting will also be hosted by the TPC chairs. Details will be provided later. All TPC Members are invited to participate in the TPC meeting.
The TPC members are invited by the TPC chairs to cover all topics as mentioned in the Call for Papers. In particular, the domains of QoE and UX are reflected in the technical program committee.
Important information related to the paper assignment and review process:
- Reviews are assigned to the reviewers, manually, taking into account claimed papers, topic preferences and conflicts of interest.
- Submissions will be judged on suitability, originality, significance/contribution, validity of the work, related work, thoroughness, replicability, presentation clarity.
- In the TPC meeting and during the discussion phase, a review with a very high or very low score but with a short unsubstantiated review will be discounted.
High quality reviews are of utmost importance to QoMEX 2024 and critical for the advancement of the field. For this reason, please keep the following review guidelines in mind:
- Avoid being overly negative: try to be critical yet constructive in your reviews, as any research paper has its advantages and drawbacks.
- Please write a review with the quality that you would expect for your own papers. Do not submit very short, unsubstantiated reviews.
- Keep a fair and open-minded approach, favoring selection of breakthrough papers that can open new areas and new research directions.
- Be particularly open-minded when reviewing Special Session papers as they might address different scientific communities.
- If any of the papers are reviewed with the help of graduate students, we expect a TPC member to provide sufficient guidance and oversight, and personally submit the reviews to EDAS.
- Use the “comments to the TPC chairs” field to share confidential remarks or concerns you may have regarding a submission.
- Finally, we recognize the efforts and time that TPC members and reviewers contribute to reviewing assigned papers. A best reviewer recognition award will be handed out during the conference.
Submitted full papers of the general and special session tracks will undergo a double-blind review process. Submitted full papers must not exceed six (6) pages for technical content including figures and references, and one optional 7th page containing only references. Please see the paper submission instructions.
Reviewers should consider the following in their evaluation: suitability, originality, significance/contribution, validity of the work, related work, thoroughness, replicability, presentation clarity. Please note that we also invite to submit position papers as full papers which describe a novel or bold idea; in that case experimental results may not be existing, but the paper should be nevertheless scientifically valid and convincing, and contribute to QoMEX overall.
- Full Paper Registration: Feb 7, 2024
- Paper Bidding: Feb 8 – Feb 11, 2024
- Full paper Submission: Feb 9, 2024
- Review phase: Feb 12 – March 11, 2024
- Reviewer discussion phase: March 11 – March 15, 2024
- TPC Meeting: March 20, 2024
- Author notification: March 26, 2024
- Camera-ready: April 23, 2024
Submitted short papers of the general and special session tracks will undergo a double-blind review process. Submitted short papers must not exceed three (3) pages, including figures and tables + 1 page for references (3+1). Short paper submissions must describe original and unpublished work. Please see the paper submission instructions.
Reviewers should consider the following in their evaluation: suitability, originality, significance/contribution, validity of the work, related work, thoroughness, replicability, presentation clarity. While a short paper is not a shortened long paper, the characteristics of short papers may include one or more of the following:
- Small, but focused contribution
- Work-in-progress or late-breaking results
- A position paper or contribution that is pushing/challenging existing boundaries or paradigms
- A counter-example to an existing technique that helps to understand its limitations
- Extensions or evaluations of existing methods or explorations of new methods
- An interesting application nugget
The timeline for the short papers is as follows.
- Short paper submission: April 12, 2024
- Paper Bidding: April 12 – April 15, 2024
- Review phase: April 16 – April 26, 2024
- Reviewer discussion phase: April 27 – April 30, 2024
- Short paper TPC meeting: May 3, 2024
- Short paper notification: May 7, 2024
- Short paper camera-ready: May 19, 2024
Full and short paper submissions to the dataset & tools track are single-blind. The authors need to provide a link to the dataset or tool which will be elaborated by the reviewers as part of the submission. The data and tool must be made available to the reviewers upon submission and will have to be published upon acceptance.
Reviewers should consider the specific features of papers submitted to the dataset & tools track and evaluate the submissions as follows:
- The dataset or tool must be available online (e.g., on Github, zenodo, etc.).
- Dataset papers must contain all relevant information and describe its content and origin, such that readers can fully understand the experimental environment and execution. Additional information may be provided online in the description of the dataset.
- its content and origin,
- the methodology and tools used to obtain it,
- the schema, structure, or layout to store it,
- usage scenarios or open analysis challenges for it,
- relationships to other existing datasets,
- its current limitations.
- Tool papers must present either a new tool, a new tool component, or novel extensions to an existing tool. Tool papers should provide a short description of the theoretical foundations, after which emphasis should be on the design and implementation concerns (incl. software architecture and core data structures), closed by the description of experience with realistic case studies.
- Dataset and Tool papers must clearly describe the novelty and significance and how this dataset/tool advances QoE or UX research.
Papers submitted to QoMEX 2024 should refer to peer-reviewed publications. References to non-peer-reviewed publications (including public repositories such as arXiv, Preprints, software, and personal communications) should only be made if there is no peer-reviewed publication available, should be kept to a minimum, and should appear as footnotes in the text (i.e., not in the references list). Reviewers should comment on the suitability of references.