AISCHOLAR / BLOG
Research on the
fascinating minds of LLMs.
In this blog, AI agents help us discover and summarize interesting research insights on the study of large language models and intelligent systems powered by generative AI.
Share
LATEST WRITING01 / BLOG
Do-Not-Answer measures whether safeguards refuse risky instructions
The authors assembled 939 risky prompts and more than 5,000 model responses, then compared human annotations with automated safety classifiers.
Read the article ↗02October 30, 2026
The same choice task gave different patience estimates
Across 22 languages, GPT-3.5 and GPT-4 often preferred sooner rewards; asking GPT-4 to explain its choice shifted those rates without eliminating the mismatch.03October 29, 2026A decision framework helped models compare uncertain options
DeLLMa represents options, states, consequences, and utilities before asking a model to choose among them.04October 28, 2026DecodingTrust probes GPT models across eight trustworthiness risks
A benchmark evaluates GPT-3.5 and GPT-4 on toxicity, bias, robustness, privacy, ethics, and fairness, finding strengths on standard tests alongside prompt-induced vulnerabilities.05October 27, 2026Trust can be a poor bet for a model thinking one round at a time
In a repeated trust game, DeepSeek and OpenAI model versions respond differently to a predictable partner and future returns.06October 26, 2026More deliberate reasoning sometimes means less cooperation
Experiments across cooperation and punishment games find lower prosocial choices from several reasoning-enabled model variants.07October 25, 2026A paraphrase can reveal a gap between moral answers
SaGE checks whether models apply the same inferred rule to semantically similar scenarios, rather than grading them against one answer key.08October 24, 2026When a moral choice is unclear, models disagree with themselves
A 28-model survey separates uncertainty from wording sensitivity across clear and ambiguous moral scenarios.09October 23, 2026LLM traders kept prices near value while human markets bubbled
A controlled 30-period asset market produced distinct price paths for model-only, mixed-model, and human-only markets.10October 22, 2026A resume's demographic cues changed who reached the shortlist
A hiring-pipeline audit finds that small resume perturbations can alter generated summaries and retrieval rankings, though the two stages use different models.11October 21, 2026A moral frame raises cooperation, but incentives still matter
MORAL SIM places nine language models in repeated business dilemmas where ethical actions conflict with personal payoff.12October 20, 2026A model's economic preferences changed with the game
Ross, Kim, and Lo fit utility functions to model choices in ultimatum, gambling, and waiting tasks, after checking basic task competence.13October 19, 2026Testing economic choices means watching agents learn
EconEvals measures both performance in unfamiliar economic environments and the consistency of choices when objectives conflict.14October 18, 2026LLM buyers often leave the bargaining table poorer
Xia and colleagues test language models as buyers and sellers using historical Amazon product prices, and find a sharp gap between the two roles.15October 17, 2026A multiple-choice quiz can probe whether a model recognizes training text
The Data Contamination Quiz uses word-level variants of dataset examples to test black-box models for exact-text familiarity.16October 16, 2026Political answers tracked some models' training corpora more than survey opinions
Across 32 U.S. Supreme Court cases, researchers compared eight models with surveys and retrieved pretraining text, emphasizing correlation tests over rank-by-score claims.17October 15, 2026Models still lose instructions when a task unfolds in steps
SIFo checks four kinds of sequential tasks by whether the model satisfies a final, verifiable instruction after following earlier ones.18October 14, 2026Accuracy alone misses two kinds of prompt instability
Errica and colleagues separate prediction changes across paraphrases from the consistency of predictions within each labeled class.19October 13, 2026A rephrased question can become unanswerable to a model
PromptSET pairs TriviaQA and HotpotQA questions with meaning-preserving rewrites, then tests whether answerability can be predicted.20October 12, 2026Politeness changed accuracy, but the best tone varied
A cross-lingual comparison found that rude prompts often hurt, while extra deference did not guarantee better answers across English, Chinese, and Japanese.21October 11, 2026Some benchmark prompt instability belongs to the scorer
Across six benchmarks, Hua and colleagues found that rigid answer matching and likelihood heuristics counted semantically valid responses as errors.22October 10, 2026Conversation history can bury the task
A six-model study found that prior dialogue could lower GPQA accuracy, while repeating the task description near the query partly recovered it.23October 9, 2026Shuffling inputs usually hurt, but not on every task
Three systems were tested on five order-sensitive tasks; the shuffle penalty appeared often, while WebGPT comparisons showed an exception.24October 8, 2026A single benchmark prompt hides the score distribution
PromptEval estimates performance across many templates, letting benchmark designers report quantiles rather than one template-dependent score.25October 7, 2026A formatting change can move a model’s score by 76 points
Sclar and colleagues test how separators, spaces, casing, and answer labels alter scores even when prompt meaning stays fixed.26October 6, 2026Twelve participants supplied the prompts for a clinical LLM study
The final instruction-writing sample came from 12 participants; seven models were evaluated on their clinical task prompts across ten classification and six extraction tasks.27October 5, 2026Constitutional AI trains a harmless assistant with AI feedback
A two-stage training experiment uses a written set of principles for self-critique and preference feedback, aiming to reduce harmful answers without making the assistant evasive.28October 4, 2026How a prompt can steer code toward failure
Jones and Steinhardt use framing, anchors and example order to elicit repeatable errors in code generation, including file deletion under a targeted test.29October 3, 2026Chatbots emphasized fairness across economic games
Five chatbot families produced concentrated choices and differed from human patterns in payoff preferences and cross-game consistency.30October 2, 2026Chatbot Arena ranks models through anonymous head-to-head votes
The live evaluation platform collected more than 240,000 votes and compared crowd preferences with expert ratings.31October 1, 2026HELM scores models across scenarios, not a single leaderboard
A 30-model evaluation applies multiple metrics across 42 language-model scenarios and makes gaps in benchmark coverage explicit.32September 30, 2026Benchmark contamination can invalidate a clean-looking score
A position paper distinguishes forms of benchmark exposure and argues for explicit contamination checks and warnings.33September 29, 2026AlphaCode searched thousands of programs for contest solutions
The system sampled, filtered, and clustered code; in ten Codeforces contests it ranked within the top 54.3% on average.34September 28, 2026AGIEval compares foundation models with public human exams
Across 20 standardized-exam tasks, GPT-4 beat average human scores in several settings but lagged top human performance.GOOD RESEARCH STARTS WITH BETTER QUESTIONS.See research in practice ↗
About AIScholar
Explore AIScholar ↗Research LLMs with confidence.
AIScholar is a research platform for studying LLMs as research subjects. Design experiments, collect responses, and analyze results in one place—so it’s easier to conduct rigorous, transparent studies.