JAMAL: A Multidimensional Benchmark for Arabic Commonsense Reasoning Across Life-Domains and Cognitive Axes

Authors

  • Basma Sayah Laboratoire d’Informatique et de Mathématiques (LIM), Amar Telidji University, Laghouat, Algeria Author
  • Dr. Attia Nehar Laboratoire d’Informatique et de Mathématiques (LIM), Amar Telidji University, Laghouat, Algeria , Computer Science Department, Ziane Achour University of Djelfa, Djelfa, Algeria Author
  • Prof. Hadda Cherroun Laboratoire d’Informatique et de Mathématiques (LIM), Amar Telidji University, Laghouat, Algeria Author
  • Dr. Slimane Bellaouar Laboratoire de Mathématiques et Sciences Appliquées (LMSA), Université de Ghardaia, Ghardaia, Algeria Author
  • Dr. Firoj Alam Qatar Computing Research Institute (QCRI), Hamad Bin Khalifa University, Doha, Qatar Author

DOI:

https://doi.org/10.66834/byjwsd23

Keywords:

Commonsense Reasoning, Arabic NLP, Language Model Evaluation, Psycholinguistically Grounded Benchmarking, WHO-ICF Framework

Abstract

Commonsense is a broad and multifaceted concept, making its evaluation a persistent challenge in natural language processing (NLP). This paper introduces JAMAL (Arabic for “Camel”), a multidimensional framework and benchmark for Arabic commonsense reasoning. JAMAL is structured along three complementary axes: (i) a life-domain axis comprising a taxonomy of 56 functional categories informed by the World Health Organization’s International Classification of Functioning, Disability, and Health (ICF), capturing diverse aspects of daily human experience; (ii) a cognitive axis organizing commonsense into three reasoning types: everyday situations, general knowledge, and problem-solving; and (iii) a cultural grounding axis distinguishing between universal, western/global, and Arabic-specific knowledge. To operationalize this framework, benchmark items are constructed using psycholinguistically inspired principles of constrained contextual prediction. We evaluate five Arabic language models using JAMAL and observe consistent differences in their performance across all axes. Notably, FANAR-27B achieves the strongest overall results among all evaluated models, outperforming FANAR-9B and smaller baselines. Overall, JAMAL provides a structured and interpretable benchmark for evaluating commonsense reasoning in Arabic, supporting the development of more robust language models through systematic analysis of their behavioral limitations.

 

Author Biographies

  • Dr. Attia Nehar, Laboratoire d’Informatique et de Mathématiques (LIM), Amar Telidji University, Laghouat, Algeria, Computer Science Department, Ziane Achour University of Djelfa, Djelfa, Algeria

    Dr. Attia Nehar is an Associate Professor at the Department of Computer Science, Ziane Achour University, Djelfa, Algeria. His research interests include machine learning, natural language processing, and Arabic text processing, with a particular focus on benchmarking and evaluating deep learning and hybrid models for linguistic and computational analysis.

  • Prof. Hadda Cherroun, Laboratoire d’Informatique et de Mathématiques (LIM), Amar Telidji University, Laghouat, Algeria

    Hadda Cherroun is a Full Professor in the Department of Computer Science at Amar Telidji University, Algeria. She holds an Engineer degree (1991), Master's degree (1997), and PhD (2007) in Computer Science, all from USTHB, Algeria. Her career began as an engineer at Amar Telidji University (1991-1997) before transitioning to an academic role as lecturer and researcher.
    %Prof. Cherroun has led numerous national and international projects, including participation in the Trans-European Mobility Programme for University Studies, the Middle East and North Africa (MEDA) project Ide@, Algeria-South Africa research cooperation initiatives and UNESCO development projects. Her research expertise spans algorithm design across multiple domains, including scheduling, compiler technology, social network analysis, and machine learning. Currently, her work focuses on AI-driven applications, particularly in NLP for Arabic language resource development, dialect-related applications, Educational Technologies, and network-based AI systems.

  • Dr. Slimane Bellaouar, Laboratoire de Mathématiques et Sciences Appliquées (LMSA), Université de Ghardaia, Ghardaia, Algeria

     Dr. Bellaouar serves as the Director of the Laboratory of Mathematics and Applied Sciences and is the Head of the Doctoral Training Committee. With a fundamental background in algorithms and computational complexity, his current research focuses on Machine Learning, Deep Learning, and Natural Language Processing (NLP), with a particular emphasis on Arabic language applications. He is also an active member of several national and international research projects. Slimane Bellaouar received his Engineering degree in Computer Science from the prestigious École nationale Supérieure d'Informatique (ESI, formerly INI) in Algiers (Algeria) in 1992. He joined the University of Ghardaïa (Algeria) as a Lecturer-researcher in 2009. In 2018, he earned his Ph.D. in computer science from the University of Laghouat (Algeria).

  • Dr. Firoj Alam, Qatar Computing Research Institute (QCRI), Hamad Bin Khalifa University, Doha, Qatar

    Dr. Firoj Alam is a Senior Scientist at the Qatar Computing Research Institute. He earned his PhD from the University of Trento, Italy, and has over a decade of experience in Artificial Intelligence, Deep/Machine Learning, Natural Language Processing, Social Media Content Analysis, Image Processing, and Conversational Analysis. He has published over 100 research papers (including 6 book chapters and 11 journal articles) in international journals, conferences, and notable workshops.

    Dr. Alam has served as the lead principal investigator for several funded projects. His current research focuses on large language models (benchmarking, native and cultural alignment), generative AI content detection, disinformation detection, fact-checking, and multimodal propaganda detection. He has also been actively contributing to building a community of researchers in these areas by organizing workshops and shared tasks. Some notable shared tasks include the CheckThat! lab at CLEF, NLP4IF 2021, and several SemEval tasks.

    He has significantly contributed to developing AI-based tools and resources to support humanitarian organizations during disaster events and aid the UN-OCHA in streamlining their Education Insecurity efforts. Dr. Alam has served as a program committee member for several top-tier conferences, including ACL, NAACL, EMNLP, ICLR, Interspeech, ICASSP, AAAI, IJCAI, and ICWSM. Additionally, he has reviewed numerous journals, including Computer Speech and Language (CSL), ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), and PLOS ONE.

References

1. T. B. Brown et al., “Language models are few-shot learners,” 2020, doi:https://doi.org/10.48550/arXiv.2005.14165.

2. OpenAI, J. Achiam, S. Adler, S. Agarwal, L. Ahmad et al., “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774, 2023, doi:https://doi.org/10.48550/arXiv.2303.08774. [Online]. Available: https://arxiv.org/abs/2303.08774

3. H. Touvron et al., “Llama 2: Open foundation and finetuned chat models,” arXiv preprint arXiv:2307.09288, 2023. [Online]. Available: https://arxiv.org/abs/2307.09288

4. J. Bai et al., “Qwen technical report,” arXiv preprint arXiv:2309.16609, 2023, doi:https://doi.org/10.48550/arXiv.2309.16609.

5. G. Team et al., “Gemini: A family of highly capable multimodal models,” 2025. [Online]. Available: https://arxiv.org/abs/2312.11805

6. R. Zellers, A. Holtzman, Y. Bisk, A. Farhadi, and Y. Choi, “HellaSwag: Can a machine really finish your sentence?” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, A. Korhonen, D. Traum, and L. Màrquez, Eds. Florence, Italy: Association for Computational Linguistics, Jul. 2019, pp. 4791–4800, doi:https://doi.org/10.18653/v1/P19-1472. [Online]. Available: https://aclanthology.org/P19-1472/ DOI: https://doi.org/10.18653/v1/P19-1472

7. D. Hendrycks et al., “Measuring massive multitask language understanding,” 2021. [Online]. Available: https://arxiv.org/abs/2009.03300

8. G. Lai, Q. Xie, H. Liu, Y. Yang, and E. Hovy, “RACE: Large-scale ReAding comprehension dataset from examinations,” in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, M. Palmer, R. Hwa, and S. Riedel, Eds. Copenhagen, Denmark: Association for Computational Linguistics, Sep. 2017, pp. 785–794, doi:https://doi.org/10.18653/v1/D1 7-1082. [Online]. Available: https://aclanthology.org/D17-1082/ DOI: https://doi.org/10.18653/v1/D17-1082

9. A. Abdelali et al., “LAraBench: Benchmarking Arabic AI with large language models,” in Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), Y. Graham and M. Purver, Eds. St. Julian’s, Malta: Association for Computational Linguistics, Mar. 2024, pp. 487–520, doi:https://do i.org/10.18653/v1/2024.eacl-long.30. [Online]. Available: https://aclanthology.org/2024.eacl-long.30/ DOI: https://doi.org/10.18653/v1/2024.eacl-long.30

10. M. T. I. Khondaker, A. Waheed, E. M. B. Nagoudi, and M. Abdul-Mageed, “GPTAraEval: A comprehensive evaluation of ChatGPT on Arabic NLP,” in Proceedings of the Conference on Empirical Methods in Natural Language Processing, H. Bouamor, J. Pino, and K. Bali, Eds. Singapore: Association for Computational Linguistics, Dec. 2023, pp. 220–247, doi:https://doi.org/10.18653/v1/2023.emnlp-main.16. [Online]. Available: https://aclanthology.org/2023.emnlp-main.16/ DOI: https://doi.org/10.18653/v1/2023.emnlp-main.16

11. F. Koto et al., “ArabicMMLU: Assessing massive multitask language understanding in Arabic,” in Findings of the Association for Computational Linguistics: ACL 2024, L.-W. Ku, A. Martins, and V. Srikumar, Eds. Bangkok, Thailand: Association for Computational Linguistics, Aug. 2024, pp. 5622–5640, doi:https://doi.org/10.18653/v1/2024.findings-acl.334. [Online]. Available: https://aclanthology.org/2024.findings-acl.334/ DOI: https://doi.org/10.18653/v1/2024.findings-acl.334

12. B. Mousi et al., “AraDiCE: Benchmarks for dialectal and cultural capabilities in LLMs,” in Proceedings of the 31st International Conference on Computational Linguistics, O. Rambow, L. Wanner, M. Apidianaki, H. Al-Khalifa, B. D. Eugenio, and S. Schockaert, Eds. Abu Dhabi, UAE: Association for Computational Linguistics, Jan. 2025, pp. 4186–4218. [Online]. Available: https://aclanthology.org/2025.coling-main.283/

13. R. N. Almatham et al., “BALSAM: A platform for benchmarking Arabic large language models,” in Proceedings of The Third Arabic Natural Language Processing Conference, K. Darwish et al., Eds. Suzhou, China: Association for Computational Linguistics, Nov. 2025, pp. 258–277, doi:https://doi.org/10.18653/v1/2025.arabicnlp-main.21. [Online]. Available: https://aclanthology.org/2025.arabicnlp-main.21/ DOI: https://doi.org/10.18653/v1/2025.arabicnlp-main.21

14. S. Al-Khalifa, N. Durrani, H. Al-Khalifa, and F. Alam, “The landscape of arabic large language models,” Communications of the ACM, vol. 68, no. 10, pp. 54–61, 2025, doi:https://doi.org/10.1145/3737453. DOI: https://doi.org/10.1145/3737453

15. K. Smith et al., “The origins of common sense in humans and machines,” in Proceedings of the Annual Meeting of the Cognitive Science Society, vol. 42, 2020, pp. 3–4. [Online]. Available: https://escholarship.org/uc/item/2367w9c4

16. S. K. Tawalbeh and M. Al-Smadi, “Is this sentence valid? an arabic dataset for commonsense validation,” CoRR, vol. abs/2008.10873, 2020. [Online]. Available: https://arxiv.org/abs/2008.10873

17. N. Sengupta et al., “Jais and jais-chat: Arabic-centric foundation and instruction-tuned open generative large language models,” 2023. [Online]. Available: https://arxiv.org/abs/2308.16149

18. M. S. Bari et al., “ALLam: Large language models for arabic and english,” in The Thirteenth International Conference on Learning Representations, 2025. [Online]. Available: https://openreview.net/forum?id=MscdsFVZrN

19. F. Team et al., “Fanar: An arabic-centric multimodal generative ai platform,” 2025. [Online]. Available: https://arxiv.org/abs/2501.13944

20. World Health Organization, International Classification of Functioning, Disability and Health (ICF). Geneva: World Health Organization, 2001. [Online]. Available: https://www.who.int/classifications/icf/en/

21. R. Zellers, Y. Bisk, R. Schwartz, and Y. Choi, “SWAG: A large-scale adversarial dataset for grounded commonsense inference,” in Proceedings of the Conference on Empirical Methods in Natural Language Processing, E. Riloff, D. Chiang, J. Hockenmaier, and J. Tsujii, Eds. Brussels, Belgium: Association for Computational Linguistics, Oct.-Nov. 2018, pp. 93–104, doi:https://doi.org/10.18653/v1/D18-1009. [Online]. Available: https://aclanthology.org/D18-1009/ DOI: https://doi.org/10.18653/v1/D18-1009

22. A. Talmor, J. Herzig, N. Lourie, and J. Berant, “CommonsenseQA: A question answering challenge targeting commonsense knowledge,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), J. Burstein, Basma Sayah et al. C. Doran, and T. Solorio, Eds. Minneapolis, Minnesota: Association for Computational Linguistics, Jun. 2019, pp. 4149–4158, doi:https://doi.org/10.18653/v1/N19-1421. [Online]. Available: https://aclanthology.org/N19-1421/

23. M. Sap, H. Rashkin, D. Chen, R. Le Bras, and Y. Choi, “Social IQa: Commonsense reasoning about social interactions,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), K. Inui, J. Jiang, V. Ng, and X. Wan, Eds. Hong Kong, China: Association for Computational Linguistics, Nov. 2019, pp. 4463–4473, doi:https://doi.org/10.18653/v1/D19-1454. [Online]. Available: https://aclanthology.org/D19-1454/ DOI: https://doi.org/10.18653/v1/D19-1454

24. Y. Bisk, R. Zellers, R. Le Bras, J. Gao, and Y. Choi, “Piqa: Reasoning about physical commonsense in natural language,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 05, 2020, pp. 7432–7439, doi:https://doi.org/10.1609/aaai.v34i05.6239. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/6239 DOI: https://doi.org/10.1609/aaai.v34i05.6239

25. M. Chen, M. D’Arcy, A. Liu, J. Fernandez, and D. Downey, “CODAH: An adversarially-authored question answering dataset for common sense,” in Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for NLP, A. Rogers, A. Drozd, A. Rumshisky, and Y. Goldberg, Eds. Minneapolis, USA: Association for Computational Linguistics, Jun. 2019, pp. 63–69, doi:https://doi.org/10.18653/v1/W19-2008. [Online]. Available: https://aclanthology.org/W19-2008/

26. B. Y. Lin, S. Lee, X. Qiao, and X. Ren, “Common sense beyond English: Evaluating and improving multilingual language models for commonsense reasoning,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), C. Zong, F. Xia, W. Li, and R. Navigli, Eds. Online: Association for Computational Linguistics, Aug. 2021, pp. 1274–1287, doi:https://doi.org/10.18653/v1/2021.acl-long.102. [Online]. Available: https://aclanthology.org/2021.acl-long.102/ DOI: https://doi.org/10.18653/v1/2021.acl-long.102

27. H. Santos, A. M. Mulvehill, K. Shen, M. Kejriwal, and D. L. McGuinness, “Tg-csr: A human-labeled dataset grounded in nine formal commonsense categories,” Data in Brief, vol. 51, p. 109666, 2023, doi:https://doi.org/10.1016/j.dib.2023.109666. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2352340923007515 DOI: https://doi.org/10.1016/j.dib.2023.109666

28. S. Saha, P. Yadav, L. Bauer, and M. Bansal, “ExplaGraphs: An explanation graph generation task for structured commonsense reasoning,” in Proceedings of the Conference on Empirical Methods in Natural Language Processing, M.-F. Moens, X. Huang, L. Specia, and S. W.-t. Yih, Eds. Online and Punta Cana, Dominican Republic: Association for Computational Linguistics, Nov. 2021, pp. 7716–7740, doi:https://doi.org/10.18653/v1/2021.emnlp-main.609. [Online]. Available: https://aclanthology.org/2021.emnlp-main.609/ DOI: https://doi.org/10.18653/v1/2021.emnlp-main.609

29. W. Zhan et al., “Score: Benchmarking long-chain reasoning in commonsense scenarios,” 2025. [Online]. Available: https://arxiv.org/abs/2503.06218

30. H. Mozannar, E. Maamary, K. El Hajal, and H. Hajj, “Neural Arabic question answering,” in Proceedings of the Fourth Arabic Natural Language Processing Workshop, W. El-Hajj et al., Eds. Florence, Italy: Association for Computational Linguistics, Aug. 2019, pp. 108–118, doi:https://doi.org/10.18653/v1/W19-4612. [Online]. Available: https://aclanthology.org/W19-4612/ DOI: https://doi.org/10.18653/v1/W19-4612

31. J. L. Lee et al., “Massively multilingual pronunciation modeling with WikiPron,” in Proceedings of the Twelfth Language Resources and Evaluation Conference, N. Calzolari et al., Eds. Marseille, France: European Language Resources Association, May 2020, pp. 4223–4228. [Online]. Available: https://aclanthology.org/2020.lrec-1.521/

32. S. Lamsiyah et al., “Arabicsense: A benchmark for evaluating commonsense reasoning in arabic with large language models,” in Proceedings of the 4th Workshop on Arabic Corpus Linguistics (WACL-4), 2025, pp. 1–11. [Online]. Available: https://aclanthology.org/2025.wacl-1.1/

33. A. Sadallah et al., “Commonsense reasoning in arab culture,” in Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2025, pp. 7695–7710, doi:https://doi.org/10.18650/ACL.2025.380. DOI: https://doi.org/10.18653/v1/2025.acl-long.380

34. A. Hornby and J. Turnbull, Oxford Advanced Learner’s Dictionary of Current English. Oxford University Press, 2015.

35. F. Ilievski, A. Oltramari, K. Ma, B. Zhang, D. L. McGuinness, and P. Szekely, “Dimensions of commonsense knowledge,” Knowledge-Based Systems, vol. 229, p. 107347, 2021, doi:https://doi.org/10.1016/j.knosys.2021. 107347. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0950705121006092 DOI: https://doi.org/10.1016/j.knosys.2021.107347

36. D. B. Lenat, “Cyc: a large-scale investment in knowledge infrastructure,” Commun. ACM, vol. 38, no. 11, p. 33–38, Nov. 1995, doi:https://doi.org/10.1145/219717.219745. DOI: https://doi.org/10.1145/219717.219745

37. M. E. Whiting and D. J. Watts, “A framework for quantifying individual and collective common sense,” Proceedings of the National Academy of Sciences, vol. 121, no. 4, p. e2309535121, 2024, doi:https://doi.org/10.1073/pnas.2309535121. DOI: https://doi.org/10.1073/pnas.2309535121

38. H. A. Simon and A. Newell, Human problem solving: The state of the theory in 1970. American Psychological Association, 1971, vol. 26, no. 2, doi:https://doi.org/10.1037/h0030806. DOI: https://doi.org/10.1037/h0030806

39. R. C. Schank and R. P. Abelson, Scripts, plans, goals, and understanding: An inquiry into human knowledge structures. Psychology press, 2013. DOI: https://doi.org/10.4324/9780203781036

40. P. Schuyler, Common Sense. Los Angeles, CA: Higher Shelf Publishing, 2003. [Online]. Available: https://www.amazon.com/Common-Sense-Peter-Schuyler/dp/1

41. R. Pesonen, “Casual reasoning: A social ecological look at human cognition and common sense,” 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:

42. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in North American Chapter of the Association for Computational Linguistics, 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:

43. A. Radford, “Improving language understanding with unsupervised learning,” OpenAI Res, 2018. [Online]. Available: https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf

44. A. Ettinger, “What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models,” Transactions of the Association for Computational Linguistics, vol. 8, pp. 34–48, 2020, doi:https://doi.org/10.1162/tacl_a_00298. [Online]. Available: https://aclanthology.org/2020.tacl-1.3/ DOI: https://doi.org/10.1162/tacl_a_00298

45. K. D. Federmeier and M. Kutas, “A rose by any other name: Long-term memory structure and sentence processing,” Journal of Memory and Language, vol. 41, no. 4, pp. 469– 495, 1999, doi:https://doi.org/10.1006/jmla.1999.2660. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0749596X99926608 DOI: https://doi.org/10.1006/jmla.1999.2660

46. K. Sakaguchi, R. L. Bras, C. Bhagavatula, and Y. Choi, “Winogrande: an adversarial winograd schema challenge at scale,” Commun. ACM, vol. 64, no. 9, p. 99–106, Aug. 2021, doi:https://doi.org/10.1145/3474381. DOI: https://doi.org/10.1145/3474381

47. R. H. Ennis, W. L. Gardiner, R. Morrow, D. Paulus, and L. Ringel, The cornell class-reasoning test, Form X, 1964.

48. S. Ouyang, J. M. Zhang, M. Harman, and M. Wang, “An empirical study of the non-determinism of chatgpt in code generation,” ACM Trans. Softw. Eng. Methodol., vol. 34, no. 2, Jan. 2025, doi:https://doi.org/10.1145/3697010. DOI: https://doi.org/10.1145/3697010

49. K. Ethayarajh, “How contextual are contextualized word representations? Comparing the geometry of BERT, ELMo, and GPT-2 embeddings,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), K. Inui, J. Jiang, V. Ng, and X. Wan, Eds. Hong Kong, China: Association for Computational Linguistics, Nov. 2019, pp. 55–65, doi:https://doi.org/10.18653/v1/D19-1006. [Online]. Available: https://aclanthology.org/D19-1006/ DOI: https://doi.org/10.18653/v1/D19-1006

50. W. Gurnee, N. Nanda, M. Pauly, K. Harvey, D. Troitskii, and D. Bertsimas, “Finding neurons in a haystack: Case studies with sparse probing,” Transactions on Machine Learning Research, 2023. [Online]. Available: https://openreview.net/forum?id=JYs1R9IMJr

51. A. Holtzman, J. Buys, L. Du, M. Forbes, and Y. Choi, “The curious case of neural text degeneration,” in International Conference on Learning Representations, 2020. [Online]. Available: https://openreview.net/forum?id=rygGQyrFvH

52. G. K. Zipf, Human Behavior and the Principle of Least Effort: An Introduction to Human Ecology. Cambridge, MA: Addison-Wesley, 1949.

53. O. Levy, Y. Goldberg, and I. Dagan, “Improving distributional similarity with lessons learned from word embeddings,” Transactions of the Association for Computational Linguistics, vol. 3, pp. 211–225, 2015, doi:https://doi.org/10.1162/tacl_a_00134. [Online]. Available: https://aclanthology.org/Q15-1016/ DOI: https://doi.org/10.1162/tacl_a_00134

54. A. Srivastava et al., “Beyond the imitation game: Quantifying and extrapolating the capabilities of language models,” Transactions on Machine Learning Research, 2023, featured Certification. [Online]. Available: https://openreview.net/forum?id=uyTL5Bvosj

55. E. M. Bender, T. Gebru, A. McMillan-Major, and S. Shmitchell, “On the dangers of stochastic parrots: Can language models be too big?,” in Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, ser. FAccT ’21. New York, NY, USA: Association for Computing Machinery, 2021, p. 610–623, doi:https://doi.org/10.1145/3442188.3445922. DOI: https://doi.org/10.1145/3442188.3445922

56. W. Antoun, F. Baly, and H. Hajj, “AraGPT2: Pretrained transformer for Arabic language generation,” in Proceedings of the Sixth Arabic Natural Language Processing Workshop, N. Habash et al., Eds. Kyiv, Ukraine (Virtual): Association for Computational Linguistics, Apr. 2021, pp. 196–207. [Online]. Available: https://aclanthology.org/2021.wanlp-1.21/

cover

Additional Files

Published

2026-06-30

Issue

Vol. 6 (2026): Issue2

Section

Full Length Articles/Research articles

License

Copyright (c) 2026 The Authors. Published by BenchCouncil Press on Behalf of International Open Benchmark Council

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

How to Cite

JAMAL: A Multidimensional Benchmark for Arabic Commonsense Reasoning Across Life-Domains and Cognitive Axes. (2026). BenchCouncil Transactions on Benchmarks, Standards and Evaluations, 6. https://doi.org/10.66834/byjwsd23