publications
Papers grouped by year in reverse chronological order.
2026
- Tech. report 2026Open and Emergent Problems in Agentic Privacy and Security: A Contextual AngleEugene Bagdasarian, Lillian Tsai, Sahar Abdelnabi, Ananth Balashankar, Borja de Balle Pigem, Sebastian Benthall, Igor Bilogrevic, Eleanor Birrell, Elie Bursztein, Kamalika Chaudhuri, Madiha Zahrah Choksi, Mihai Christodorescu, Rachel Cummings, Adam Davies, Dj Dvijotham, Seliem El-Sayed, Ferdinando Fioretto, Matt Franchi, Julien Freudiger, Adria Gascon, Roxana Geambasu, Sahra Ghalebikesabi, Hamed Haddadi, Jamie Hayes, Ashish Hooda, Amir Houmansadr, Somesh Jha, Peter Kairouz, Chloé Kiddon, Tadayoshi Kohno, Abdullatif Köksal, Alison Lentz, Haoran Li, Tianshi Li, Nathan Malkin, Sarah Meiklejohn, Niloofar Mireshghallah, Sewoong Oh, Katherine Ortiz, Ivan Petrov, Francesco Pinto, Daniel Ramage, Mariana Raykova, Franziska Roesner, Edo Roth, Jacqueline Rowe, Khawaja Shams, Reza Shokri, Ilia Shumailov, Yan Shvartzshnaider, Dawn Song, Jose Such, Octavian Suciu, Pierre Tholoniat, Matt Tierney, Trishita Tiwari, Hal Triedman, Ren Yi, Wen Zhang, Xuhui Zhou, Kassem Fawaz, Stefan Mellem, Helen Nissenbaum, Nina Taft, Sarah de Haas, and Marco Gruteser
The vision of an ecosystem of general and highly capable autonomous agents challenges accepted principles of secure and privacy-preserving system engineering. Inspired by the theory of Contextual Integrity, we argue that in order for agents to act appropriately, their behaviors should comply with societal norms and expectations. This manuscript explores contextual approaches to engineering agentic systems that embody an understanding of societal norms and uphold the appropriateness of actions by design. It presents a list of key open research problems in this area to create awareness among the broader research community across academia, government, and industry.
@techreport{Fioretto:CAPS26, title = {Open and Emergent Problems in Agentic Privacy and Security: A Contextual Angle}, year = {2026}, institution = {Google}, url = {https://research.google/pubs/open-and-emergent-problems-in-agentic-privacy-and-security-a-contextual-angle/}, author = {Bagdasarian, Eugene and Tsai, Lillian and Abdelnabi, Sahar and Balashankar, Ananth and {de Balle Pigem}, Borja and Benthall, Sebastian and Bilogrevic, Igor and Birrell, Eleanor and Bursztein, Elie and Chaudhuri, Kamalika and Choksi, Madiha Zahrah and Christodorescu, Mihai and Cummings, Rachel and Davies, Adam and Dvijotham, Dj and El-Sayed, Seliem and Fioretto, Ferdinando and Franchi, Matt and Freudiger, Julien and Gascon, Adria and Geambasu, Roxana and Ghalebikesabi, Sahra and Haddadi, Hamed and Hayes, Jamie and Hooda, Ashish and Houmansadr, Amir and Jha, Somesh and Kairouz, Peter and Kiddon, Chloé and Kohno, Tadayoshi and Köksal, Abdullatif and Lentz, Alison and Li, Haoran and Li, Tianshi and Malkin, Nathan and Meiklejohn, Sarah and Mireshghallah, Niloofar and Oh, Sewoong and Ortiz, Katherine and Petrov, Ivan and Pinto, Francesco and Ramage, Daniel and Raykova, Mariana and Roesner, Franziska and Roth, Edo and Rowe, Jacqueline and Shams, Khawaja and Shokri, Reza and Shumailov, Ilia and Shvartzshnaider, Yan and Song, Dawn and Such, Jose and Suciu, Octavian and Tholoniat, Pierre and Tierney, Matt and Tiwari, Trishita and Triedman, Hal and Yi, Ren and Zhang, Wen and Zhou, Xuhui and Fawaz, Kassem and Mellem, Stefan and Nissenbaum, Helen and Taft, Nina and {de Haas}, Sarah and Gruteser, Marco}, note = {Google technical report} } - preprint 2026Posterior Flow MatchingChenrui Ma, Xi Xiao, Lin Zhao, Xinru Jiang, Tianyang Wang, Yanning Shen, and Ferdinando Fioretto
Flow matching learns how to transport noise toward data, but its mean prediction does not directly report the endpoint uncertainty remaining at an intermediate state. This paper introduces Posterior Flow Matching (PFM), a general generative modeling framework that learns conditional velocity means and componentwise variances with a Gaussian posterior model. The same model evaluation supplies both a transport prediction and endpoint uncertainty, making posterior diagnostics available throughout generation. These learned moments support a mean-driven ordinary differential equation (ODE) and a posterior-driven stochastic differential equation (SDE), whose local noise scales adapt to the predicted uncertainty. Across image, point-cloud, molecular, and action generation, PFM improves generation quality and provides endpoint-posterior readouts throughout generation.
@article{Fioretto:PFM26, title = {Posterior Flow Matching}, year = {2026}, journal = {CoRR}, doi = {10.48550/arXiv.2610.13994}, author = {Ma, Chenrui and Xiao, Xi and Zhao, Lin and Jiang, Xinru and Wang, Tianyang and Shen, Yanning and Fioretto, Ferdinando}, volume = {abs/2610.13994} } - LHMP 2026Yufei Zhu, Jinhao Liang, Sven Koenig, Ferdinando Fioretto, and Martin Magnusson
@inproceedings{Fioretto:LHMP26, title = {Plausible and Feasible Long-Term Human Trajectory Prediction via Motion Field-Regularized Flow Matching}, year = {2026}, url = {https://motionpredictionicra2026.github.io/}, author = {Zhu, Yufei and Liang, Jinhao and Koenig, Sven and Fioretto, Ferdinando and Magnusson, Martin}, booktitle = {8th Workshop on Long-term Human Motion Prediction at ICRA}, } - preprint 2026Survey Design for Fair Resource AllocationAbdellah Aznag, Max Biggs, Ferdinando Fioretto, and Joonhyuk Ko
Many large-scale public-funding programs allocate resources to groups based on the collective needs of individuals within the group, yet these needs are generally unknown in advance and must be estimated, often using costly surveys. We study a survey-design problem: with a fixed survey budget, how should samples be allocated across groups so that the eventual resource allocation maximizes social welfare? We show that several commonly used allocation rules, ranging from winner-take-all, to proportional to needs, to equal split, all correspond to the class of alpha–fair, or isoelastic, social welfare functions. Under this objective, the regret of a plug-in allocation based on estimated needs corresponds to a scaled power divergence on allocation shares, containing the squared-Hellinger distance (alpha=1/2) and, as alpha goes to 1, the Kullback–Leibler divergence. A second-order expansion of this regret yields an explicit decision-aware sampling rule that tilts classical Neyman sampling by a share-correction factor, routing more samples to small-share groups whose score misestimation is most costly. For count-valued needs, such as eligible-population counts, under proportional allocation (alpha=1/2) and common low-eligibility, small-share conditions, the rule reduces to sampling proportional to the square root of the known group population size, independent of the unknown positive-need counts. To make the rule operational without prior knowledge of group scores, we propose a two-stage adaptive design and derive, through a second-order analysis, a universal C^2/3 rate at which the pilot budget should grow relative to the total sampling budget C. Experiments on the Low Income Home Energy Assistance Program and on Title I funding using the U.S. Census SAIPE data demonstrate substantial reductions in misallocation regret relative to uniform and Neyman benchmarks
@article{Fioretto:SurveyDesign26, title = {Survey Design for Fair Resource Allocation}, author = {Aznag, Abdellah and Biggs, Max and Fioretto, Ferdinando and Ko, Joonhyuk}, year = {2026}, doi = {10.2139/ssrn.7555080}, journal = {SSRN}, volume = {7555080} } - AI4ChipDesign 2026Tommaso Dreossi, Christopher M. Bryant, Hao Liu, Nathan Mirman, Noah Kessler, Michael Frei, Kaustubh Pabba, Harish Krishnaswamy, Jacob K. Christopher, and Ferdinando Fioretto
@inproceedings{Fioretto:AI4ChipDesign26, title = {Constrained Diffusion for Inverse Design of Fabricable Multi-Layer RF Passives from Partial S-Parameter Specifications}, author = {Dreossi, Tommaso and Bryant, Christopher M. and Liu, Hao and Mirman, Nathan and Kessler, Noah and Frei, Michael and Pabba, Kaustubh and Krishnaswamy, Harish and Christopher, Jacob K. and Fioretto, Ferdinando}, booktitle = {AI for Chip Design Workshop at NeurIPS}, year = {2026}, note = {Accepted; oral presentation}, url = {https://openreview.net/forum?id=kDIc133oYt}, } - preprint 2026Simulator-Refined Diffusion for Radio-Frequency Inverse DesignJinhao Liang, Jacob K. Christopher, Michael Frei, Tommaso Dreossi, and Ferdinando Fioretto
Diffusion models have shown potential in inverse design of printed circuit boards (PCBs), enabling the generation of layouts conditioned on target S-parameters. Despite this promise, applying diffusion models to PCB layout generation remains challenging due to their difficulty in meeting the quantitative electromagnetic specifications. A common approach is gradient-based guidance, which biases the diffusion sampling process with the gradient of an objective used for evaluation. However, full-wave electromagnetic simulators are accurate but expensive and typically non-differentiable, whereas differentiable surrogates are informative but not always reliable. To address these limitations, this paper proposes Simulator-Refined Diffusion (SRD), a novel combination of a low-fidelity differentiable surrogate and a high-fidelity non-differentiable simulator within the diffusion sampling process. Unlike standard zeroth-order optimization, which requires a great number of random perturbations, our approach uses the surrogate’s gradient to propose the perturbation direction while the simulator then searches based on this direction to identify an effective design update. Experimental results across different settings show that this method consistently outperforms current state-of-the-art methods, producing layouts whose simulated S-parameters match the target specifications up to 21.2% closer for in-distribution targets and up to 19.8% for out-of-distribution targets.
@article{fioretto:26p, title = {Simulator-Refined Diffusion for Radio-Frequency Inverse Design}, author = {Liang, Jinhao and Christopher, Jacob K. and Frei, Michael and Dreossi, Tommaso and Fioretto, Ferdinando}, journal = {CoRR}, year = {2026}, volume = {abs/2609.38363}, doi = {10.48550/arXiv.2609.38363}, } - preprint 2026Michael Cardei, Anjun Hu, Chenrui Ma, Francesco Pinto, Chawin Sitawarin, Chulin Xie, and Ferdinando Fioretto
@article{fioretto:26o, title = {Only What You've Signed up For: Secure Workflows with Constrained Diffusion Agents}, author = {Cardei, Michael and Hu, Anjun and Ma, Chenrui and Pinto, Francesco and Sitawarin, Chawin and Xie, Chulin and Fioretto, Ferdinando}, journal = {CoRR}, year = {2026}, volume = {abs/2610.15177}, doi = {10.48550/arXiv.2610.15177}, } - preprint 2026Michael Y. Fatemi, Jinhao Liang, and Ferdinando Fioretto
@article{fioretto:26n, title = {Training-Free Diffusion Planning with Analytical Local Scores}, author = {Fatemi, Michael Y. and Liang, Jinhao and Fioretto, Ferdinando}, journal = {CoRR}, year = {2026}, volume = {abs/2610.01959}, doi = {10.48550/arXiv.2610.01959}, } - preprint 2026SEABench: Benchmarking Endogenous Misalignment In Self-Evolving AgentsSaswat Das, Parvati Viswanathan, Daniel Donnelly, Chang Huang, Sahar Abdelnabi, and Ferdinando Fioretto
Self-evolving LLM agents have gained prominence for their ability to improve after deployment by modifying their harness, including their controller instructions, memory management protocols, and reusable tools and skills, in response to user and environment feedback. However, locally useful updates may persist into later tasks where they produce unsafe behavior, even without direct adversarial influence. To study this risk, we introduce SEABench, a benchmark for studying endogenous misalignment arising from agent self-evolution, with 48 longitudinal task sequences that span multiple evolution surfaces, task domains, and harm types in a rich personal-assistant environment. To account for the stochasticity inherent in agentic operations, we provide an adaptive trajectory discovery pipeline that probes for failures while preserving original task intent and supports causal attribution through paired non-evolving agents and attribution scores. Our evaluation across multiple recent LLMs, evolution surfaces, and harm types reveals that self-evolution indeed increases task completion rates but often at the cost of safety failures that are absent for paired non-evolving baseline agents. We also show that qualitatively different safety behaviors emerge across evolution surfaces and harm types. Further, we show that this divergence in safety behavior is reflected in agents’ chain-of-thought reasoning, which yields an effective monitoring strategy that can mitigate unsafe behavior with a low false positive rate.
@article{fioretto:26m, title = {SEABench: Benchmarking Endogenous Misalignment In Self-Evolving Agents}, author = {Das, Saswat and Viswanathan, Parvati and Donnelly, Daniel and Huang, Chang and Abdelnabi, Sahar and Fioretto, Ferdinando}, journal = {CoRR}, year = {2026}, volume = {abs/2609.35596}, doi = {10.48550/arXiv.2609.35596}, } - preprint 2026Constrained Graph Diffusion for Mixed Integer OptimizationVincenzo Di Vito, Yusuf Guven, Babak Badnava, Deepjyoti Deka, Kaarthik Sundar, and Ferdinando Fioretto
This paper proposes a novel learning-based approach to approximately solve instances of mixed-integer optimization problems. These problems are computationally challenging, as they require jointly determining discrete and continuous decisions while satisfying complex combinatorial constraints. We introduce Constrained Graph Diffusion (CGD), a learning-based framework that approximately solves recurring instances of such problems by learning a conditional distribution over their discrete decisions. CGD uses a graph-based diffusion model and incorporates constraint information directly into the reverse diffusion process, steering intermediate predictions toward the feasible region throughout generation. By operating on continuous relaxations of the discrete variables, CGD defines a differentiable constrained generation pathway up to terminal discrete recovery. Once the discrete decision is recovered and fixed, a numerical optimizer solves the remaining continuous problem, avoiding online combinatorial search over the binary variables while retaining numerical optimization for continuous completion. We evaluate CGD on AC-OPF with branch switching and discrete portfolio optimization, demonstrating substantial improvements in feasibility and solution quality over learning-based baselines while achieving speedups of up to 543\times over state-of-the-art MIP solvers on large instances.
@article{fioretto:26i, title = {Constrained Graph Diffusion for Mixed Integer Optimization}, author = {Di Vito, Vincenzo and Guven, Yusuf and Badnava, Babak and Deka, Deepjyoti and Sundar, Kaarthik and Fioretto, Ferdinando}, year = {2026}, volume = {abs/2608.13079}, journal = {CoRR}, doi = {10.48550/arXiv.2608.13079} } - preprint 2026Constraint-Aware Flow Matching: Decision Aligned End-to-End Training for Constrained SamplingJacob K. Christopher, James E. Warner, and Ferdinando Fioretto
Deep generative models provide state-of-the-art performance across a wide array of applications, with recent studies showing increasing applicability for science and engineering. Despite a growing corpus of literature focused on the integration of physics-based constraints into the generation process, existing approaches fail to enforce strict constraint satisfaction while maintaining sample quality. In particular, training-free constrained sampling methods, while providing per-sample feasibility guarantees, introduce a fundamental mismatch between the training objective and the constrained sampling procedure, often leading to performance degradation. Identifying this training-sampling misalignment as a central limitation of current constrained generative modeling approaches, this paper proposes Constraint-Aware Flow Matching, a novel end-to-end framework that explicitly incorporates constraint projections into the training objective. By aligning the model’s learned dynamics with the constrained sampling process, the proposed method mitigates distributional shift induced by projection-based corrections, enabling high-quality constrained generation. The proposed approach is evaluated on three challenging real-world benchmarks, illustrating the generality and efficacy of the method.
@article{fioretto:26e, title = {Constraint-Aware Flow Matching: Decision Aligned End-to-End Training for Constrained Sampling}, author = {Christopher, Jacob K. and Warner, James E. and Fioretto, Ferdinando}, journal = {CoRR}, year = {2026}, volume = {abs/2605.12754}, } - preprint 2026Simple Self-Conditioning Adaptation for Masked Diffusion ModelsMichael Cardei, Huu Binh Ta, and Ferdinando Fioretto
Masked diffusion models (MDMs) generate discrete sequences by iterative denoising under an absorbing masking process. In standard masked diffusion, if a token remains masked after a reverse update, the model discards its clean-state prediction for that position. Thus, still-masked positions must be repeatedly inferred from the mask token alone. This design choice limits cross-step refinement. To address this limitation, this paper proposes a simple, yet effective, post-training adaptation for MDMs that conditions each denoising step on the model’s own previous clean-state predictions. The resulting method, called Self-Conditioned Masked Diffusion Models (SCMDM), requires minimal architectural change, does not introduce a recurrent latent-state pathway, does not rely on an auxiliary reference model, and adds no extra denoiser evaluations during sampling. This is an important departure from partial self-conditioning approaches which requires expensive model training from scratch. In particular, the paper shows that partial self-conditioning, including the commonly used 50% dropout strategy for training self-conditioned models from scratch, is suboptimal in the post-training regime. Instead, once the model’s self-generated clean-state estimates become informative, the specialization to refinement is preferable to mixing conditional and unconditional objectives. SCMDM is evaluated across multiple domains, demonstrating consistent improvement over vanilla MDM baselines, achieving nearly a 50% reduction in generative perplexity on OWT-trained models (42.89 to 23.72), alongside strong improvements in discretized image synthesis quality, small molecular generation, and enhanced fidelity in genomic distribution modeling.
@article{Fioretto:26d, author = {Cardei, Michael and Ta, Huu Binh and Fioretto, Ferdinando}, title = {Simple Self-Conditioning Adaptation for Masked Diffusion Models}, year = {2026}, journal = {CoRR}, volume = {abs/2604.26985}, } - preprint 2026NeuroFilter: Activation-Based Guardrails for Privacy-Conscious LLM AgentsSaswat Das, and Ferdinando Fioretto
Agentic Large Language Models (LLMs) are models able to reason, plan, and execute tools over unstructured data. These abilities enable transformative applications in domains spanning personal assistants, financial, and legal domains, etc. While these systems can substantially improve productivity and service quality, effective agency typically requires access to sensitive personal or organizational information. However, this access introduces critical inference-time privacy risks, specifically regarding contextually appropriate information disclosure. While recent studies highlight the inability of agentic LLMs to consistently adhere to privacy norms, existing defenses often rely on auxiliary LLM-based monitors. However, these defenses are expensive and offer limited protection against attacks that are robust to semantic censorship. To contrast this background, this paper proposes a notion of privacy filters based on activation probing. We show that these filters are both computationally efficient and effective for both single-turn and multi-turn conversational settings. Furthermore, this work provides the first systematic investigation into probing model internals across a conversation trajectory, moving beyond static, single-prompt analysis to capture the evolving state of privacy-sensitive interactions, illustrating their applicability across contexts, attack styles, multiple base models, and challenging settings such as mosaic attacks, while providing nuanced insights into the nature of the discovered concepts and modular construction of probes.
@article{Fioretto:26a, title = {NeuroFilter: Activation-Based Guardrails for Privacy-Conscious LLM Agents}, author = {Das, Saswat and Fioretto, Ferdinando}, year = {2026}, journal = {CoRR}, volume = {abs/2601.14660}, doi = {10.48550/arXiv.2601.14660}, } - NeurIPS 2026Chance-constrained Flow Matching for High-Fidelity Constraint-aware GenerationJinhao Liang, Yixuan Sun, Anirban Samaddar, Sandeep Madireddy, and Ferdinando FiorettoFPI Workshop, MLxOR Workshop, and COML Workshop (NeurIPS 2025)
Generative models excel at synthesizing high-fidelity samples from complex data distributions, but they often violate hard constraints arising from physical laws or task specifications. A common remedy is to project intermediate samples onto the feasible set; however, repeated projection can distort the learned distribution and induce a mismatch with the data manifold. Thus, recent multi-stage procedures attempt to defer projection to clean samples during sampling, but they increase algorithmic complexity and accumulate errors across steps. This paper addresses these challenges by proposing a novel training-free method, Chance-constrained Flow Matching (CCFM), that integrates stochastic optimization into the sampling process, enabling effective enforcement of hard constraints while maintaining high-fidelity sample generation. Importantly, CCFM guarantees feasibility in the same manner as conventional repeated projection, yet, despite operating directly on noisy intermediate samples, it is theoretically equivalent to projecting onto the feasible set defined by clean samples. This yields a sampler that mitigates distributional distortion. Empirical experiments show that CCFM outperforms current state-of-the-art constrained generative models in modeling complex physical systems governed by partial differential equations and molecular docking problems, delivering higher feasibility and fidelity.
@inproceedings{Fioretto:NeurIPS26f, title = {Chance-constrained Flow Matching for High-Fidelity Constraint-aware Generation}, author = {Liang, Jinhao and Sun, Yixuan and Samaddar, Anirban and Madireddy, Sandeep and Fioretto, Ferdinando}, booktitle = {Advances in Neural Information Processing Systems}, year = {2026}, doi = {10.48550/arXiv.2509.25157}, note = {Accepted}, } - NeurIPS 2026Simulation-Informed Diffusion for Decentralized Multi-robot Motion PlanningJinhao Liang, Sven Koenig, and Ferdinando Fioretto
Decentralized multi-robot motion planning requires each robot to generate collision-free trajectories from local observations, without global sensing or reliable communication. However, most existing planners, whether classical or learning-based, generate trajectories from a static snapshot of the local observation, which limits their ability to anticipate the future behavior of neighboring robots. This limitation is critical as the number of robots increases and the environment becomes more cluttered. To overcome this challenge, this paper introduces Simulation-Informed Diffusion (SID), a decentralized framework built on constraint-aware diffusion models (CADM). SID first uses CADM to simulate the future trajectories of neighboring robots from their currently observed states, and then uses the same CADM to plan each robot’s own trajectory under safety constraints informed by these simulations. Crucially, the accurate simulation of neighbors enables a minimal communication scheme that triggers coordination only when necessary in highly congested scenarios. Experiments across diverse environments show that SID consistently outperforms baseline methods in terms of planning effectiveness and constraint satisfaction, and scales to scenarios with 108 robots and 160 obstacles.
@inproceedings{fioretto:NeurIPS26e, title = {Simulation-Informed Diffusion for Decentralized Multi-robot Motion Planning}, author = {Liang, Jinhao and Koenig, Sven and Fioretto, Ferdinando}, booktitle = {Advances in Neural Information Processing Systems}, year = {2026}, note = {Accepted}, } - NeurIPS 2026Drift Flow MatchingChenrui Ma, Xi Xiao, Lin Zhao, Tianyang Wang, Ferdinando Fioretto, and Yanning Shen
Iterative generative models such as Flow Matching and Diffusion models have demonstrated strong test-time scaling behavior, where additional inference computation can improve generation quality. In contrast, Drift Models offer efficient one-step generation, but their direct generation paradigm limits such flexibility. In this work, we propose Drift Flow Matching (DFM), a framework that connects drifting generative modeling with flow-based iterative generation. DFM preserves the efficiency of direct transport maps while enabling generation to be refined through multiple inference steps when desired. This bridges the gap between one-step Drift Models and multi-step Flow Matching methods, and provides a novel generative paradigm that can adapt sampling computation to different quality–efficiency requirements. Extensive experiments across different tasks and datasets demonstrate the effectiveness and generality of the proposed framework.
@inproceedings{fioretto:NeurIPS26d, title = {Drift Flow Matching}, author = {Ma, Chenrui and Xiao, Xi and Zhao, Lin and Wang, Tianyang and Fioretto, Ferdinando and Shen, Yanning}, booktitle = {Advances in Neural Information Processing Systems}, year = {2026}, note = {Accepted} } - NeurIPS 2026Constrained Code Generation with Discrete DiffusionLize Shao, Michael Cardei, Zichen Xie, Ferdinando Fioretto, and Wenxi Wang
Discrete diffusion models are a powerful, emerging paradigm for code generation. They construct programs through iterative refinement of partially corrupted token sequences and enable parallel token refinement. Importantly, this paradigm exposes a global program state at each denoising step, which provides a natural intervention point for enforcing program-level functionality and security constraints, guiding the generation before the final code is committed. Building on this observation, the paper introduces Constrained Diffusion for Code (CDC), a training-free neurosymbolic inference framework that integrates constraint satisfaction directly into the reverse denoising process. CDC augments the base discrete diffusion sampler with constraint-aware denoising operators that combine mathematical optimization with program analysis to identify constraint-relevant regions of the intermediate program state and locally adjust the denoising trajectory, steering generation toward feasible programs while remaining close to the base model. Across code generation benchmarks, CDC consistently improves constraint satisfaction in functional correctness, security, and even syntax, outperforming discrete diffusion and autoregressive baselines with less corrective computation and more localized edits.
@inproceedings{fioretto:NeurIPS26c, title = {Constrained Code Generation with Discrete Diffusion}, author = {Shao, Lize and Cardei, Michael and Xie, Zichen and Fioretto, Ferdinando and Wang, Wenxi}, booktitle = {Advances in Neural Information Processing Systems}, year = {2026}, note = {Accepted}, } - NeurIPS 2026Colosseum: Auditing Collusion in Cooperative Multi-Agent SystemsMason Nakamura, Abhinav Kumar, Saswat Das, Sahar Abdelnabi, Saaduddin Mahmud, Ferdinando Fioretto, Shlomo Zilberstein, and Eugene BagdasarianIn Advances in Neural Information Processing Systems, 2026Accepted; Evaluations and Datasets Track|Abs|Bib|AlphaXiv
Multi-agent systems, where LLM agents communicate through free-form language, enable sophisticated coordination for solving complex cooperative tasks. This surfaces a unique safety problem when individual agents form a coalition and \emphcollude to pursue secondary goals and degrade the joint objective. In this paper, we present Colosseum, a framework for auditing LLM agents’ collusive behavior in multi-agent settings. We ground how agents cooperate through a Distributed Constraint Optimization Problem (DCOP) and measure collusion via regret relative to the cooperative optimum. Colosseum tests each LLM for collusion under different objectives, persuasion tactics, and network topologies. Through our audit, we show that most out-of-the-box models exhibited a propensity to collude when a secret communication channel was artificially formed. Furthermore, we discover “collusion on paper” when agents plan to collude in text but would often pick non-collusive actions, thus providing little effect on the joint task. Colosseum provides a new way to study collusion by measuring communications and actions in rich yet verifiable environments.
@inproceedings{Fioretto:NeurIPS26b, title = {Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems}, author = {Nakamura, Mason and Kumar, Abhinav and Das, Saswat and Abdelnabi, Sahar and Mahmud, Saaduddin and Fioretto, Ferdinando and Zilberstein, Shlomo and Bagdasarian, Eugene}, booktitle = {Advances in Neural Information Processing Systems}, year = {2026}, doi = {10.48550/arXiv.2602.15198}, note = {Accepted; Evaluations and Datasets Track}, } - NeurIPS 2026Search-Augmented Masked Diffusion Models for Constrained GenerationHuu Binh Ta, Michael Cardei, Alvaro Velasquez, and Ferdinando Fioretto
Discrete diffusion models generate sequences by iteratively denoising samples corrupted by categorical noise, offering an appealing alternative to autoregressive decoding for structured and symbolic generation. However, standard training targets a likelihood-based objective that primarily matches the data distribution and provides no native mechanism for enforcing hard constraints or optimizing non-differentiable properties at inference time. This work addresses this limitation and introduces Search-Augmented Masked Diffusion (SearchDiff), a training-free neurosymbolic inference framework that integrates informed search directly into the reverse denoising process. At each denoising step, the model predictions define a proposal set that is optimized under a user-specified property satisfaction, yielding a modified reverse transition that steers sampling toward probable and feasible solutions. Experiments in biological design and symbolic reasoning illustrate that SearchDiff substantially improves constraint satisfaction and property adherence, while consistently outperforming discrete diffusion and autoregressive baselines.
@inproceedings{Fioretto:NeurIPS26a, title = {Search-Augmented Masked Diffusion Models for Constrained Generation}, author = {Ta, Huu Binh and Cardei, Michael and Velasquez, Alvaro and Fioretto, Ferdinando}, booktitle = {Advances in Neural Information Processing Systems}, year = {2026}, doi = {10.48550/arXiv.2602.02727}, note = {Accepted}, } - UncertaiNLP 2026From Positionwise Confidence to Prefix Scheduling: Verifier Skipping in Speculative DecodingHaoxuan Luo, Jameson Sandler, and Ferdinando FiorettoIn ACL 2026 Workshop on Uncertainty in Natural Language Processing (UncertaiNLP2026), 2026|Abs|Bib|AlphaXiv
Speculative decoding is a leading technique to reduce the cost of autoregressive generation by using a small drafter to propose several tokens, which are then verified in parallel by a larger target model. Speculative diffusion decoding (SDD) further removes sequential drafting by generating every position in a draft block in parallel with a discrete diffusion model. However, SDD still invokes the target on every block, leaving verification as a potential bottleneck. This paper recognizes that this creates a new control handle: whether to invoke the verifier at all. Thus, we study verifier skipping, a lossy policy that commits a selected draft prefix directly, and ask which confidence signal should schedule it. Interestingly, our study finds that better token predictors need not yield better schedulers: skips require contiguous high-confidence prefixes, while short skips can induce additional drafting rounds. To study this mismatch, we compare raw confidence with learned marginal and conditional survival scores under the same policy, using Strict SDD, lenience, and top-k acceptance as baselines. On HumanEval with DiffuCoder-7B-Instruct and Qwen3-32B, all three confidence signals save 9.6% to 13.5% of verifier calls at the same observed pass@1 as Strict SDD. Surprisingly, raw confidence saves the most; marginal survival has higher positionwise AUROC than raw confidence at most positions, yet neither learned signal dominates online. Our analysis shows that verifier skipping is a useful new lossy axis and, surprisingly, its key challenge is prefix scheduling rather than token prediction alone.
@inproceedings{fioretto:ACL26ws, title = {From Positionwise Confidence to Prefix Scheduling: Verifier Skipping in Speculative Decoding}, author = {Luo, Haoxuan and Sandler, Jameson and Fioretto, Ferdinando}, year = {2026}, booktitle = {ACL 2026 Workshop on Uncertainty in Natural Language Processing (UncertaiNLP2026)}, volume = {abs/2608.14787}, } - SEGAN 2026Stability-constrained AC optimal power flow–A Gaussian process-based approachVincenzo Di Vito, Kaarthik Sundar, Ferdinando Fioretto, and Deepjyoti Deka
The Alternating Current Optimal Power Flow (ACOPF) problem is a core task in power system operations, aimed at determining cost-effective generation dispatch while satisfying physical and operational constraints. However, conventional ACOPF formulations rely on steady-state models and neglect the dynamic behavior of generators, which can lead to operating points that are economically optimal but dynamically unstable. This paper proposes a novel, data-driven approach to incorporate generator dynamics into the ACOPF using Gaussian Process (GP) models. Specifically, it introduces an exponential surrogate function to characterize the stability of solutions to the differential equations governing synchronous generator dynamics. The exponent, which indicates whether system trajectories decay (stable) or grow (unstable), is learned as a function of the bus voltage using GP regression. Crucially, the framework enables probabilistic stability assessment to be integrated directly into the optimization process. The resulting dynamics-aware ACOPF formulation identifies operating points that satisfy both operational safety and dynamic stability criteria. Numerical experiments on the IEEE 39-bus, 57-bus, and 118-bus systems demonstrate that the proposed method efficiently captures generator dynamics using limited training data, leading to more reliable and robust decisions across a wide range of operating conditions.
@article{Fioretto:SEGAN, title = {Stability-constrained AC optimal power flow--A Gaussian process-based approach}, author = {Di Vito, Vincenzo and Sundar, Kaarthik and Fioretto, Ferdinando and Deka, Deepjyoti}, year = {2026}, journal = {Sustainable Energy, Grids and Networks}, volume = {46}, doi = {10.1016/j.segan.2026.102300}, month = jun, pages = {102300}, url = {https://doi.org/10.1016/j.segan.2026.102300} } - TSG 2026Global-Decision-Focused Neural ODEs for Proactive Grid Resilience ManagementShuyi Chen, Ferdinando Fioretto, Feng Qiu, and Shixiang Zhu
Extreme hazard events such as wildfires and hurricanes increasingly threaten power systems, causing widespread outages and disrupting critical services. To mitigate these risks, grid operators must act proactively—pre-positioning crews and resources or scheduling hardening activities—guided by forecasts that directly support operational objectives. Traditionally, utilities and agencies have followed a predict-then-optimize paradigm: first generating impact forecasts, then using them to inform response planning. However, this two-step approach inherently separates forecasting from decision-making, overlooking how forecast errors propagate into downstream actions and often producing misaligned or suboptimal plans. We address this gap with predict-all-then-optimize-globally (PATOG), a framework that unifies outage prediction with globally optimized interventions. At its core, our global-decision-focused (GDF) Neural ODE model jointly captures outage dynamics and optimizes pre-event resilience strategies in a decision-aware manner. Unlike conventional methods, our approach ensures spatially and temporally coherent decisions, enhancing both predictive accuracy and operational efficiency. Evaluations on synthetic and real-world data show that GDF reduces decision regret by up to 75% compared to two-stage baselines, enabling more effective proactive planning.
@article{Fioretto:tsg26, author = {Chen, Shuyi and Fioretto, Ferdinando and Qiu, Feng and Zhu, Shixiang}, title = {Global-Decision-Focused Neural ODEs for Proactive Grid Resilience Management}, journal = {IEEE Transactions on Smart Grids}, volume = {}, number = {}, pages = {1--1}, year = {2026}, url = {https://ieeexplore.ieee.org/document/11297004}, doi = {10.1109/TSG.2025.3642407}, } - EPSR 2026Learning to optimize meets neural-ODE: Real-time, stability-constrained AC OPFVincenzo Di Vito, Mostafa Mohammadian, Kyri Baker, and Ferdinando Fioretto
Recent developments in applying machine learning to address Alternating Current Optimal Power Flow (AC OPF) problems have demonstrated significant potential in providing close to optimal solutions for generator dispatch in near real-time. While these learning to optimize (LtO) methods have demonstrated remarkable performance on steady-state operations, practical applications often demand compliance with dynamic constraints when used for fast-timescale optimization. This paper addresses this gap and develops a real-time stability-constrained OPF model (DynOPF-Net) that simultaneously addresses both optimality and dynamical stability within learning-assisted grid operations. The model is a unique integration of learning to optimize that learns a mapping from load conditions to OPF solutions, capturing the OPF’s physical and engineering constraints, with Neural Ordinary Differential Equations, capturing generator dynamics, enabling the inclusion of a subset of stability constraints. Numerical results on the WSCC 9-bus and IEEE 118-bus benchmark systems demonstrate that DynOPF-Net can produce highly accurate AC-OPF solutions while also ensuring system stability, contrasting the unstable results obtained by state-of-the-art LtO methods.
@article{Fioretto:epsr26, title = {Learning to optimize meets neural-ODE: Real-time, stability-constrained AC OPF}, author = {{Di Vito}, Vincenzo and Mohammadian, Mostafa and Baker, Kyri and Fioretto, Ferdinando}, journal = {Electric Power Systems Research}, volume = {258}, pages = {113071}, year = {2026}, issn = {0378-7796}, doi = {https://doi.org/10.1016/j.epsr.2026.113071}, url = {https://www.sciencedirect.com/science/article/pii/S0378779626003640}, } - CVPR 2026FREESTYLE: An Anchor-Free Mechanism for Training-Free Style-Aligned Image GenerationMinseok Oh, Jihun Park, Jongmin Gim, Minwoo Choi, Kyoungmin Lee, Ferdinando Fioretto, and Sunghoon Im
Text-to-Image (T2I) generation models have become central to creative workflows, where producing style-consistent image sets is crucial for applications such as visual identity design, illustration, and asset creation. While recent training-free methods have enabled efficient style-aligned synthesis, they commonly rely on anchor-dependent propagation, where style features from a single reference image are shared across the batch. This dependency often leads to inconsistent results: when the anchor fails to capture the intended reference style, its unintended artifacts are propagated to all samples, and excessive feature sharing further causes content leakage and semantic distortion. To overcome these limitations, we propose FREESTYLE, a training-free and anchor-free framework for robust style alignment within a scale-wise autoregressive generation paradigm. Our method integrates three core components—Majority Voting (MV), which aggregates dominant style cues across the batch to form a representative style feature; Majority Style Injection (MSI), which adaptively injects these aggregated features to enforce global style coherence; and Set-Based Projection (SBP), which refines local object regions by projecting them onto a shared style manifold for context-aware adaptation. Extensive experiments demonstrate that, without any retraining or parameter updates, FREESTYLE achieves state-of-the-art performance in both style consistency and content preservation, while maintaining real-time inference efficiency.
@inproceedings{Fioretto:CVPR26, author = {Oh, Minseok and Park, Jihun and Gim, Jongmin and Choi, Minwoo and Lee, Kyoungmin and Fioretto, Ferdinando and Im, Sunghoon}, title = {FREESTYLE: An Anchor-Free Mechanism for Training-Free Style-Aligned Image Generation}, booktitle = {Computer Vision and Pattern Recognition}, month = jun, year = {2026}, pages = {3779-3788}, } - ICLR 2026Constrained Diffusion for Protein Design with Hard Structural ConstraintsJacob K Christopher, Austin Seamann, Jingyi Cui, Sagar Khare, and Ferdinando Fioretto
Diffusion models offer a powerful means of capturing the manifold of realistic protein structures, enabling rapid design for protein engineering tasks. However, existing approaches observe critical failure modes when precise constraints are necessary for functional design. To this end, we present a constrained diffusion framework for structure-guided protein design, ensuring strict adherence to functional requirements while maintaining precise stereochemical and geometric feasibility. The approach integrates proximal feasibility updates with ADMM decomposition into the generative process, scaling effectively to the complex constraint sets of this domain. We evaluate on challenging protein design tasks, including motif scaffolding and vacancy-constrained pocket design, while introducing a novel curated benchmark dataset for motif scaffolding in the PDZ domain. Our approach achieves state-of-the-art, providing perfect satisfaction of bonding and geometric constraints with no degradation in structural diversity.Competing Interest StatementThe authors have declared no competing interest.
@inproceedings{Fioretto:ICLR26a, author = {Christopher, Jacob K and Seamann, Austin and Cui, Jingyi and Khare, Sagar and Fioretto, Ferdinando}, title = {Constrained Diffusion for Protein Design with Hard Structural Constraints}, year = {2026}, doi = {10.1101/2025.10.15.682365}, booktitle = {International Conference on Learning Representations}, volume = {14}, url = {https://www.biorxiv.org/content/early/2025/10/15/2025.10.15.682365} } - ICLR 2026Gen-DFL: Decision-Focused Generative Learning for Robust Decision MakingPrince Zizhuang Wang, Jinhao Liang, Shuyi Chen, Ferdinando Fioretto, and Shixiang Zhu
Decision-focused learning (DFL) integrates predictive models with downstream optimization, directly training machine learning models to minimize decision errors. While DFL has been shown to provide substantial advantages when compared to a counterpart that treats the predictive and prescriptive models separately, it has also been shown to struggle in high-dimensional and risk-sensitive settings, limiting its applicability in real-world settings. To address this limitation, this paper introduces decision-focused generative learning (Gen-DFL), a novel framework that leverages generative models to adaptively model uncertainty and improve decision quality. Instead of relying on fixed uncertainty sets, Gen-DFL learns a structured representation of the optimization parameters and samples from the tail regions of the learned distribution to enhance robustness against worst-case scenarios. This approach mitigates over-conservatism while capturing complex dependencies in the parameter space. The paper shows, theoretically, that Gen-DFL achieves improved worst-case performance bounds compared to traditional DFL. Empirically, it evaluates Gen-DFL on various scheduling and logistics problems, demonstrating its strong performance against existing DFL methods.
@inproceedings{Fioretto:ICLR26b, title = {Gen-DFL: Decision-Focused Generative Learning for Robust Decision Making}, author = {Wang, Prince Zizhuang and Liang, Jinhao and Chen, Shuyi and Fioretto, Ferdinando and Zhu, Shixiang}, year = {2026}, booktitle = {International Conference on Learning Representations}, volume = {14}, doi = {10.48550/arXiv.2502.05468}, } - MLSys 2026SpecDiff-2: Scaling Diffusion Drafter Alignment For Faster Speculative DecodingoralJameson Sandler, Jacob K Christopher, Thomas Hartvigsen, and Ferdinando Fioretto
Speculative decoding has become the standard approach for accelerating Large Language Model (LLM) inference. It exploits a lossless draft-then-verify procedure to circumvent the latency of autoregressive decoding, achieving impressive speed-ups. Yet, current speculative decoding approaches remain limited by two fundamental bottlenecks: (1) the autoregressive dependency during drafting which limits parallelism, and (2) frequent rejections of draft tokens caused by misalignment between the draft and verify models. This paper proposes SpecDiff-2, a novel framework to jointly address these two bottlenecks. It leverages discrete diffusion as a non-autoregressive drafter to address bottleneck (1) and develops novel techniques to calibrate discrete diffusion drafters with autoregressive verifiers, addressing bottleneck (2). Experimental results across a comprehensive benchmark suite show that SpecDiff-2 achieves a new state-of-the-art across reasoning, coding, and mathematical benchmarks, improving tokens-per-second by up to an average of +55% over previous baselines and obtaining up to 5.5x average speed-up over standard decoding, without any loss of accuracy.
@inproceedings{Fioretto:MLSys26, author = {Sandler, Jameson and Christopher, Jacob K and Hartvigsen, Thomas and Fioretto, Ferdinando}, title = {SpecDiff-2: Scaling Diffusion Drafter Alignment For Faster Speculative Decoding}, year = {2026}, booktitle = {Conference on Machine Learning and Systems}, volume = {9}, } - L4DC 2026Learning to Solve Constrained Bilevel Control Co-Design ProblemsOralJames Kotary, Himanshu Sharma, Ethan King, Draguna L Vrabie, Ferdinando Fioretto, and Jan DrgonaIn Proceedings of The 8th Annual Learning for Dynamics and Control Conference, 2026|Abs|Bib|PDF|Online|AlphaXiv
Learning to Optimize (L2O) is a subfield of machine learning (ML) in which ML models are trained to solve parametric optimization problems. The general goal is to learn a fast approximator of solutions to constrained optimization problems, as a function of their defining parameters. Prior L2O methods focus almost entirely on single-level programs, in contrast to the bilevel programs, whose constraints are themselves expressed in terms of optimization subproblems. Bilevel programs have numerous important use cases but are notoriously difficult to solve, particularly under stringent time demands. This paper proposes a framework for learning to solve a broad class of challenging bilevel optimization problems, by leveraging modern techniques for differentiation through optimization problems. The framework is illustrated on an array of synthetic bilevel programs, as well as challenging control system co-design problems, showing how neural networks can be trained as efficient approximators of parametric bilevel optimization.
@inproceedings{Fioretto:L4DC26, title = {Learning to Solve Constrained Bilevel Control Co-Design Problems}, author = {Kotary, James and Sharma, Himanshu and King, Ethan and Vrabie, Draguna L and Fioretto, Ferdinando and Drgona, Jan}, year = {2026}, booktitle = {Proceedings of The 8th Annual Learning for Dynamics and Control Conference}, volume = {331}, doi = {10.48550/arXiv.2507.09050}, pages = {222--241}, series = {Proceedings of Machine Learning Research}, publisher = {PMLR}, } - SatML 2026SoK: Data Minimization in Machine LearningRobin Staab, Nikola Jovanović, Kimberly Mai, Prakhar Ganesh, Martin Vechev, Ferdinando Fioretto, and Matthew Jagielski
Data minimization (DM) describes the principle of collecting only the data strictly necessary for a given task. It is a foundational principle across major data protection regulations like GDPR and CPRA. Violations of this principle have substantial real-world consequences, with regulatory actions resulting in fines reaching hundreds of millions of dollars. Notably, the relevance of data minimization is particularly pronounced in machine learning (ML) applications, which typically rely on large datasets, resulting in an emerging research area known as Data Minimization in Machine Learning (DMML). At the same time, existing work on other ML privacy and security topics often addresses concerns relevant to DMML without explicitly acknowledging the connection. This disconnect leads to confusion among practitioners, complicating their efforts to implement DM principles and interpret the terminology, metrics, and evaluation criteria used across different research communities. To address this gap, our work introduces a comprehensive framework for DMML, including a unified data pipeline, adversaries, and points of minimization. This framework allows us to systematically review the literature on data minimization and DM-adjacent methodologies, for the first time presenting a structured overview designed to help practitioners and researchers effectively apply DM principles. Our work facilitates a unified DM-centric understanding and broader adoption of data minimization strategies in AI/ML.
@inproceedings{Fioretto:SatML26, title = {SoK: Data Minimization in Machine Learning}, author = {Staab, Robin and Jovanović, Nikola and Mai, Kimberly and Ganesh, Prakhar and Vechev, Martin and Fioretto, Ferdinando and Jagielski, Matthew}, year = {2026}, booktitle = {IEEE Secure and Trustworthy Machine Learning Conference}, volume = {3}, doi = {}, } - AAAI 2026Discrete-Guided Diffusion for Scalable and Safe Multi-Robot Motion PlanningJinhao Liang, Sven Koenig, and Ferdinando Fioretto
Multi-Robot Motion Planning (MRMP) involves generating collision-free trajectories for multiple robots operating in a shared continuous workspace. While discrete multi-agent path finding (MAPF) methods are broadly adopted due to their scalability, their coarse discretization severely limits trajectory quality. In contrast, continuous optimization-based planners offer higher-quality paths but suffer from the curse of dimensionality, resulting in poor scalability with respect to the number of robots. This paper tackles the limitations of these two approaches by introducing a novel framework that integrates discrete MAPF solvers with constrained generative diffusion models. The resulting framework, called Discrete-Guided Diffusion (DGD), has three key characteristics: (1) it decomposes the original nonconvex MRMP problem into tractable subproblems with convex configuration spaces, (2) it combines discrete MAPF solutions with constrained optimization techniques to guide diffusion models capture complex spatiotemporal dependencies among robots, and (3) it incorporates a lightweight constraint repair mechanism to ensure trajectory feasibility. The proposed method sets a new state-of-the-art performance in large-scale, complex environments, scaling to 100 robots while achieving planning efficiency and high success rates.
@inproceedings{Fioretto:AAAI26, title = {Discrete-Guided Diffusion for Scalable and Safe Multi-Robot Motion Planning}, author = {Liang, Jinhao and Koenig, Sven and Fioretto, Ferdinando}, year = {2026}, booktitle = {AAAI Conference on Artificial Intelligence}, volume = {40}, doi = {10.48550/arXiv.2508.20095}, } - AAAI 2026A Unified Framework for Prompt Privacy is Elusive and MisleadingPrakhar Ganesh, Yash More, Marco Romanelli, Ferdinando Fioretto, and Golnoosh Farnadi
As large language models (LLMs) become increasingly integrated into daily life, privacy concerns are on the rise. Driven by the appeal of convenient, universal solutions, current practices reflect a drift toward a one-size-fits-all approach to privacy. Unfortunately, we argue that a unified framework for prompt privacy is elusive and can instead mislead, creating even greater risk due to a false sense of safety. We identify five desirable properties of an effective privacy framework for prompts, namely protection guarantees, performance, efficiency, domain adaptability, and user accessibility, and highlight that existing frameworks, such as sanitization, differential privacy, cryptography, and contextual integrity, only satisfy a subset of these properties. Beyond individual frameworks, we find underlying tensions between these properties that preclude the development of a unified framework. We recommend two critical paths forward: emphasizing context-specific and application-specific evaluation of privacy frameworks, and fostering user awareness and privacy literacy.
@inproceedings{Fioretto:PerFM26, title = {A Unified Framework for Prompt Privacy is Elusive and Misleading}, author = {Ganesh, Prakhar and More, Yash and Romanelli, Marco and Fioretto, Ferdinando and Farnadi, Golnoosh}, year = {2026}, booktitle = {PerFM Workshop - at AAAI}, volume = {1}, }
2025
- AFME 2025Miriam Rateike, Awa Dieng, Jamelle Watson-Daniels, Ferdinando Fioretto, and Golnoosh FarnadiIn Proceedings of the Algorithmic Fairness Through the Lens of Metrics and Evaluation, 2025Workshop report|Bib|PDF|Online
@inproceedings{Fioretto:AFME25, title = {Algorithmic Fairness Through the Lens of Metrics and Evaluation (AFME) 2024}, author = {Rateike, Miriam and Dieng, Awa and Watson-Daniels, Jamelle and Fioretto, Ferdinando and Farnadi, Golnoosh}, booktitle = {Proceedings of the Algorithmic Fairness Through the Lens of Metrics and Evaluation}, pages = {1--7}, year = {2025}, volume = {279}, series = {Proceedings of Machine Learning Research}, publisher = {PMLR}, note = {Workshop report} } - Tech. report 2025Ferdinando Fioretto, and Bhavya Kailkhura2025Lawrence Livermore National Laboratory white paper, LLNL-CONF-871579; 2025 Workshop on Envisioning Frontiers in AI and Computing for Biological Research|Bib|PDF|Online
@techreport{Fioretto:LLNL25, title = {How can we ground Frontiers Models for Biological Research to Domain Knowledge, First Principles, and Constraints?}, author = {Fioretto, Ferdinando and Kailkhura, Bhavya}, institution = {Lawrence Livermore National Laboratory}, number = {LLNL-CONF-871579}, year = {2025}, url = {https://www.osti.gov/biblio/2511362}, note = {Lawrence Livermore National Laboratory white paper, LLNL-CONF-871579; 2025 Workshop on Envisioning Frontiers in AI and Computing for Biological Research} } - PESGM 2025Jinhao Liang, Ferdinando Fioretto, and Chenye Wu
@inproceedings{Fioretto:PESGM25, title = {Robust Bidding Strategies in Local Energy Markets}, author = {Liang, Jinhao and Fioretto, Ferdinando and Wu, Chenye}, year = {2025}, booktitle = {2025 IEEE Power \& Energy Society General Meeting (PESGM)}, publisher = {IEEE}, pages = {1--5}, doi = {10.1109/PESGM52009.2025.11225045}, url = {https://ieeexplore.ieee.org/document/11225045}, } - preprint 2025Optimal Allocation of Privacy Budget on Hierarchical Data ReleaseJoonhyuk Ko, Juba Ziani, and Ferdinando Fioretto
Releasing useful information from datasets with hierarchical structures while preserving individual privacy presents a significant challenge. Standard privacy-preserving mechanisms, and in particular Differential Privacy, often require careful allocation of a finite privacy budget across different levels and components of the hierarchy. Sub-optimal allocation can lead to either excessive noise, rendering the data useless, or to insufficient protections for sensitive information. This paper addresses the critical problem of optimal privacy budget allocation for hierarchical data release. It formulates this challenge as a constrained optimization problem, aiming to maximize data utility subject to a total privacy budget while considering the inherent trade-offs between data granularity and privacy loss. The proposed approach is supported by theoretical analysis and validated through comprehensive experiments on real hierarchical datasets. These experiments demonstrate that optimal privacy budget allocation significantly enhances the utility of the released data and improves the performance of downstream tasks.
@article{Fioretto:PrivacyBudget25, title = {Optimal Allocation of Privacy Budget on Hierarchical Data Release}, author = {Ko, Joonhyuk and Ziani, Juba and Fioretto, Ferdinando}, year = {2025}, doi = {10.48550/arXiv.2505.10871}, journal = {CoRR}, volume = {abs/2505.10871} } - preprint 2025Learning to Solve Optimization Problems Constrained with Partial Differential EquationsYusuf Guven, Vincenzo Di Vito, and Ferdinando Fioretto
Partial differential equation (PDE)-constrained optimization arises in many scientific and engineering domains, such as energy systems, fluid dynamics and material design. In these problems, the decision variables (e.g., control inputs or design parameters) are tightly coupled with the PDE state variables, and the feasible set is implicitly defined by the governing PDE constraints. This coupling makes the problems computationally demanding, as it requires handling high dimensional discretization and dynamic constraints. To address these challenges, this paper introduces a learning-based framework that integrates a dynamic predictor with an optimization surrogate. The dynamic predictor, a novel time-discrete Neural Operator (Lu et al.), efficiently approximate system trajectories governed by PDE dynamics, while the optimization surrogate leverages proxy optimizer techniques (Kotary et al.) to approximate the associated optimal decisions. This dual-network design enables real-time approximation of optimal strategies while explicitly capturing the coupling between decisions and PDE dynamics. We validate the proposed approach on benchmark PDE-constrained optimization tasks inlacing Burgers’ equation, heat equation and voltage regulation, and demonstrate that it achieves solution quality comparable to classical control-based algorithms, such as the Direct Method and Model Predictive Control (MPC), while providing up to four orders of magnitude improvement in computational speed.
@article{Fioretto:25i, title = {Learning to Solve Optimization Problems Constrained with Partial Differential Equations}, author = {Guven, Yusuf and Vito, Vincenzo Di and Fioretto, Ferdinando}, year = {2025}, journal = {CoRR}, volume = {abs/2509.24573}, doi = {10.48550/arXiv.2509.24573}, } - preprint 2025The Disparate Impacts of Speculative DecodingJameson Sandler, Ahmet Üstün, Marco Romanelli, Sara Hooker, and Ferdinando Fioretto
The practice of speculative decoding, whereby inference is probabilistically supported by a smaller, cheaper, “drafter” model, has become a standard technique for systematically reducing the decoding time of large language models. This paper conducts an analysis of speculative decoding through the lens of its potential disparate speed-up rates across tasks. Crucially, the paper shows that speed-up gained from speculative decoding is not uniformly distributed across tasks, consistently diminishing for under-fit, and often underrepresented tasks. To better understand this phenomenon, we derive an analysis to quantify this observed “unfairness” and draw attention to the factors that motivate such disparate speed-ups to emerge. Further, guided by these insights, the paper proposes a mitigation strategy designed to reduce speed-up disparities and validates the approach across several model pairs, revealing on average a 12% improvement in our fairness metric.
@article{Fioretto:25h, title = {The Disparate Impacts of Speculative Decoding}, author = {Sandler, Jameson and Üstün, Ahmet and Romanelli, Marco and Hooker, Sara and Fioretto, Ferdinando}, year = {2025}, journal = {CoRR}, volume = {abs/2510.02128}, doi = {10.48550/arXiv.2510.02128}, } - preprint 2025Beyond Jailbreaking: Auditing Contextual Privacy in LLM AgentsSaswat Das, Jameson Sandler, and Ferdinando FiorettoAlso appeared in: MTI-LLM Workshop (NeurIPS 2025)
Large Language Model agents have begun to appear as personal assistants, customer service bots, and clinical aides. While these applications deliver substantial operational benefits, they also require continuous access to sensitive data, which increases the likelihood of unauthorized disclosures. This study proposes an auditing framework for conversational privacy that quantifies and audits these risks. The proposed Conversational Manipulation for Privacy Leakage (CMPL) framework, is an iterative probing strategy designed to stress-test agents that enforce strict privacy directives. Rather than focusing solely on a single disclosure event, CMPL simulates realistic multi-turn interactions to systematically uncover latent vulnerabilities. Our evaluation on diverse domains, data modalities, and safety configurations demonstrate the auditing framework’s ability to reveal privacy risks that are not deterred by existing single-turn defenses. In addition to introducing CMPL as a diagnostic tool, the paper delivers (1) an auditing procedure grounded in quantifiable risk metrics and (2) an open benchmark for evaluation of conversational privacy across agent implementations.
@article{Fioretto:25g, title = {Beyond Jailbreaking: Auditing Contextual Privacy in LLM Agents}, author = {Das, Saswat and Sandler, Jameson and Fioretto, Ferdinando}, year = {2025}, journal = {CoRR}, volume = {abs/2506.10171}, doi = {10.48550/arXiv.2506.10171}, } - Book 2025Differential Privacy in Artificial Intelligence: From Theory to PracticeFerdinando Fioretto, Pascal Van Hentenryck, and others
Differential Privacy in Artificial Intelligence: From Theory to Practice is a comprehensive resource designed to review the principles and applications of differential privacy in a world increasingly driven by data. This book delves into the theoretical underpinnings of differential privacy, its use in machine learning systems, practical implementation details, and its broader social and legal ramifications. Intended as a primer and a deep dive, it lays a solid foundation by introducing essential concepts and mechanisms critical to understanding differential privacy. From theoretical foundations to practical application, the book is organized into five distinct parts. Part I reviews the foundational notions of differential privacy in the central and local models, delving into composition and privacy amplification. The discussion extends to practical strategies for data release and the creation of synthetic data, which is essential for real-world applications. Part II focuses on the application of differential privacy in optimization and learning, examining the integration of privacy measures in machine learning, including private optimization methods and private federated learning. Beyond technical applications, the book highlights the use of differential privacy in critical sectors such as healthcare and energy, and discusses its implications in image and video analysis in Part III. Part IV provides a thorough look at the tools and challenges in deploying privacy-preserving models, including insights into programming frameworks and machine learning tools. Finally, Part V addresses the societal impact of differential privacy, discussing its intersection with public policy, law, fairness, and bias. Targeted at researchers, practitioners, and policymakers, Differential Privacy in Artificial Intelligence: From Theory to Practice aims to be an essential guide for anyone committed to advancing privacy in the digital age, providing the knowledge needed to develop and deploy effective and ethical privacy solutions across various domains.
@book{fioretto:DP_book25, title = {Differential Privacy in Artificial Intelligence: From Theory to Practice}, author = {Fioretto, Ferdinando and Hentenryck, Pascal Van and others}, year = {2025}, editora = {Ferdinando Fioretto}, editorb = {Pascal Van Hentenryck}, publisher = {Now Publishers, Inc.}, journal = {Now Publishers, Inc.}, url = {https://www.nowpublishers.com/article/BookDetails/9781638284765} } - Book-Ch 2025Differential Privacy: Overview and Fundamental TechniquesFerdinando Fioretto, Pascal Van Hentenryck, and Juba ZianiIn Differential Privacy in Artificial Intelligence: From Theory to Practice, 2025|Abs|Bib|Online|AlphaXiv
This chapter is meant to be part of the book "Differential Privacy in Artificial Intelligence: From Theory to Practice" and provides an introduction to Differential Privacy. It starts by illustrating various attempts to protect data privacy, emphasizing where and why they failed, and providing the key desiderata of a robust privacy definition. It then defines the key actors, tasks, and scopes that make up the domain of privacy-preserving data analysis. Following that, it formalizes the definition of Differential Privacy and its inherent properties, including composition, post-processing immunity, and group privacy. The chapter also reviews the basic techniques and mechanisms commonly used to implement Differential Privacy in its pure and approximate forms.
@incollection{Fioretto:DP_book_ch1, title = {Differential Privacy: Overview and Fundamental Techniques}, author = {Fioretto, Ferdinando and Hentenryck, Pascal Van and Ziani, Juba}, booktitle = {Differential Privacy in Artificial Intelligence: From Theory to Practice}, editor = {Fioretto, Ferdinando and Hentenryck, Pascal Van}, publisher = {Now Publishers, Inc.}, doi = {10.48550/arXiv.2411.04710}, year = {2025}, pages = {1-41}, chapter = {1}, url = {https://www.nowpublishers.com/article/Chapter/9781638284765?cId=978-1-63828-477-2.ch1} } - IISE 2025Raed Al Kontar, Tianbao Yang, Ferdinando Fioretto, and Farzad Yousefian
@article{AlKontar25, author = {Al Kontar, Raed and Yang, Tianbao and Fioretto, Ferdinando and Yousefian, Farzad}, title = {Editorial: Federated Distributed Learning and Analytics}, journal = {IISE Transactions}, volume = {57}, number = {7}, pages = {741--742}, year = {2025}, publisher = {Taylor \& Francis}, doi = {10.1080/24725854.2025.2479047}, url = {https://doi.org/10.1080/24725854.2025.2479047}, eprint = {https://doi.org/10.1080/24725854.2025.2479047} } - NeurIPS 2025Training-Free Constrained Generation With Stable Diffusion ModelsSpotlightStefano Zampini, Jacob K Christopher, Luca Oneto, Davide Anguita, and Ferdinando Fioretto
Stable diffusion models represent the state-of-the-art in data synthesis across diverse domains and hold transformative potential for applications in science and engineering, e.g., by facilitating the discovery of novel solutions and simulating systems that are computationally intractable to model explicitly. However, their current utility in these fields is severely limited by an inability to enforce strict adherence to physical laws and domain-specific constraints. Without this grounding, the deployment of such models in critical applications, ranging from material science to safety-critical systems, remains impractical. This paper addresses this fundamental limitation by proposing a novel approach to integrate stable diffusion models with constrained optimization frameworks, enabling them to generate outputs that satisfy stringent physical and functional requirements. We demonstrate the effectiveness of this approach through material science experiments requiring adherence to precise morphometric properties, inverse design problems involving the generation of stress-strain responses using video generation with a simulator in the loop, and safety settings where outputs must avoid copyright infringement.
@inproceedings{Fioretto:NeurIPS25a, title = {Training-Free Constrained Generation With Stable Diffusion Models}, author = {Zampini, Stefano and Christopher, Jacob K and Oneto, Luca and Anguita, Davide and Fioretto, Ferdinando}, booktitle = {Advances in Neural Information Processing Systems}, year = {2025}, volume = {38}, pages = {}, publisher = {Curran Associates, Inc.}, } - NeurIPS 2025Constrained Discrete DiffusionMichael Cardei, Jacob K Christopher, Thomas Hartvigsen, Bhavya Kailkhura, and Ferdinando FiorettoBioSafe GenAI Workshop (NeurIPS 2025)
Constraints are critical in text generation as LLM outputs are often unreliable when it comes to ensuring generated outputs adhere to user defined instruction or general safety guidelines. To address this gap, we present Constrained Discrete Diffusion (CDD), a novel method for enforcing constraints on natural language by integrating discrete diffusion models with differentiable optimization. Unlike conventional text generators, which often rely on post-hoc filtering or model retraining for controllable generation, we propose imposing constraints directly into the discrete diffusion sampling process. We illustrate how this technique can be applied to satisfy a variety of natural language constraints, including (i) toxicity mitigation by preventing harmful content from emerging, (ii) character and sequence level lexical constraints, and (iii) novel molecule sequence generation with specific property adherence. Experimental results show that our constraint-aware procedure achieves high fidelity in meeting these requirements while preserving fluency and semantic coherence, outperforming auto-regressive and existing discrete diffusion approaches.
@inproceedings{Fioretto:NeurIPS25b, title = {Constrained Discrete Diffusion}, author = {Cardei, Michael and Christopher, Jacob K and Hartvigsen, Thomas and Kailkhura, Bhavya and Fioretto, Ferdinando}, booktitle = {Advances in Neural Information Processing Systems}, year = {2025}, volume = {38}, pages = {}, publisher = {Curran Associates, Inc.}, } - CDC 2025Privacy-Preserving Convex Optimization: When Differential Privacy Meets Stochastic ProgrammingVladimir Dvorkin, Ferdinando Fioretto, Pascal Van Hentenryck, Pierre Pinson, and Jalal Kazempour
Convex optimization finds many real-life applications, where - optimized on real data - optimization results may expose private data attributes (e.g., individual health records, commercial information, etc.), thus leading to privacy breaches. To avoid these breaches and formally guarantee privacy to optimization data owners, we develop a new privacy-preserving perturbation strategy for convex optimization programs by combining stochastic (chance-constrained) programming and differential privacy. Unlike standard noise-additive strategies, which perturb either optimization data or optimization results, we express the optimization variables as functions of the random perturbation using linear decision rules; we then optimize these rules to accommodate the perturbation within the problem’s feasible region by enforcing chance constraints. This way, the perturbation is feasible and makes different, yet adjacent in the sense of a given distance function, optimization datasets statistically similar in randomized optimization results, thereby enabling probabilistic differential privacy guarantees. The chance-constrained optimization additionally internalizes the conditional value-at-risk measure to model the tolerance towards the worst-case realizations of the optimality loss with respect to the non-private solution. We demonstrate the privacy properties of our perturbation strategy analytically and through optimization and machine learning applications.
@inproceedings{Fioretto:CDC25, title = {Privacy-Preserving Convex Optimization: When Differential Privacy Meets Stochastic Programming}, author = {Dvorkin, Vladimir and Fioretto, Ferdinando and Hentenryck, Pascal Van and Pinson, Pierre and Kazempour, Jalal}, year = {2025}, booktitle = {IEEE Conference on Decision and Control (CDC)}, publisher = {}, } - ICML 2025Simultaneous Multi-Robot Motion Planning with Projected Diffusion ModelsJinhao Liang, Jacob Christopher, Sven Koenig, and Ferdinando Fioretto
Recent advances in diffusion models hold significant potential in robotics, enabling the generation of diverse and smooth trajectories directly from raw representations of the environment. Despite this promise, applying diffusion models to motion planning remains challenging due to their difficulty in enforcing critical constraints, such as collision avoidance and kinematic feasibility. These limitations become even more pronounced in Multi-Robot Motion Planning (MRMP), where multiple robots must coordinate in shared spaces. To address these challenges, this work proposes Simultaneous MRMP Diffusion (SMD), a novel approach integrating constrained optimization into the diffusion sampling process to produce collision-free, kinematically feasible trajectories. Additionally, the paper introduces a comprehensive MRMP benchmark to evaluate trajectory planning algorithms across scenarios with varying robot densities, obstacle complexities, and motion constraints. Experimental results show SMD consistently outperforms classical and other learning-based motion planners, achieving higher success rates and efficiency in complex multi-robot environments.
@inproceedings{Fioretto:ICML25, title = {Simultaneous Multi-Robot Motion Planning with Projected Diffusion Models}, author = {Liang, Jinhao and Christopher, Jacob and Koenig, Sven and Fioretto, Ferdinando}, year = {2025}, booktitle = {International Conference on Machine Learning}, publisher = {}, } - NeuS 2025Neuro-symbolic Generative Diffusion Models for Physically Grounded, Robust, and Safe GenerationDARPA Disruptive IdeaJacob K. Christopher, Michael Cardei, Jinhao Liang, and Ferdinando Fioretto
Despite the remarkable generative capabilities of diffusion models, their integration into safety-critical or scientifically rigorous applications remains hindered by the need to ensure compliance with stringent physical, structural, and operational constraints. To address this challenge, this paper introduces Neuro-Symbolic Diffusion (NSD), a novel framework that interleaves diffusion steps with symbolic optimization, enabling the generation of certifiably consistent samples under user-defined functional and logic constraints. This key feature is provided for both standard and discrete diffusion models, enabling, for the first time, the generation of both continuous (e.g., images and trajectories) and discrete (e.g., molecular structures and natural language) outputs that comply with constraints. This ability is demonstrated on tasks spanning three key challenges: (1) Safety, in the context of non-toxic molecular generation and collision-free trajectory optimization; (2) Data scarcity, in domains like drug discovery and materials engineering; and (3) Out-of-domain generalization, where enforcing symbolic constraints allows adaptation beyond the training distribution.
@inproceedings{Fioretto:NeuS25, title = {Neuro-symbolic Generative Diffusion Models for Physically Grounded, Robust, and Safe Generation}, author = {Christopher, Jacob K. and Cardei, Michael and Liang, Jinhao and Fioretto, Ferdinando}, year = {2025}, booktitle = {International Conference on Neuro-symbolic Systems (NeuS)}, } - FAccT 2025The Data Minimization Principle in Machine LearningFerdinando Fioretto, Prakhar Ganesh, Cuong Tran, and Reza ShokriWorkshop on Generative AI and Law (ICML 2024)
The principle of data minimization requires organizations to collect, process, and retain only personal data that is adequate, relevant, and limited to what is necessary for specified objectives. It’s grounded in the expectation that not all collected data is essential for the objective and, instead, contributes to a heightened risk of information leakage. However, despite its legal significance and endorsement by global data protection regulations, the data minimization principle lacks a mathematical formalization suitable for real-world ML applications. In particular, the current discourse on data minimization practices often overlooks two crucial aspects: \bf (1) the individualized nature of minimization (e.g., information that is unimportant for an individual may be critical for another) and \bf (2) its intrinsic link to data privacy. To overcome these limitations, this paper introduces a formal framework for data minimization in ML while being faithful to its legal notion; adapts and evaluates various optimization algorithms to solve the problem of data minimization; and analyzes their compatibility with real-world privacy. In particular, we seek to answer a critical question: “Do data minimization requirements in various regulations meet privacy expectations in legal frameworks?” Our evaluations reveal that the answer is, unfortunately, no. While being an implicit intention, the requirements of data minimization are not necessarily aligned with risk of reconstruction and re-identification and thus may not provide the expected privacy protection. With this work, we aim to lay down a path for future research for developing ML systems that comply with the legal requirements of data minimization.
@inproceedings{Fioretto:FAccT25, title = {The Data Minimization Principle in Machine Learning}, author = {Fioretto, Ferdinando and Ganesh, Prakhar and Tran, Cuong and Shokri, Reza}, booktitle = {ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT)}, year = {2025}, } - ICLR 2025Learning To Solve Differential Equation Constrained Optimization ProblemsSpotlightVincenzo Di Vito, Mostafa Mohammadian, Kyri Baker, and Ferdinando Fioretto
Differential equations (DE) constrained optimization plays a critical role in numerous scientific and engineering fields, including energy systems, aerospace engineering, ecology, and finance, where optimal configurations or control strategies must be determined for systems governed by ordinary or stochastic differential equations. Despite its significance, the computational challenges associated with these problems have limited their practical use. To address these limitations, this paper introduces a learning-based approach to DE-constrained optimization that combines techniques from proxy optimization and neural differential equations. The proposed approach uses a dual-network architecture, with one approximating the control strategies, focusing on steady-state constraints, and another solving the associated DEs. This combination enables the approximation of optimal strategies while accounting for dynamic constraints in near real-time. Experiments across problems in energy optimization and finance modeling show that this method provides full compliance with dynamic constraints and it produces results up to 25 times more precise than other methods which do not explicitly model the system’s dynamic equations.
@inproceedings{Fioretto:ICLR25, title = {Learning To Solve Differential Equation Constrained Optimization Problems}, author = {Vito, Vincenzo Di and Mohammadian, Mostafa and Baker, Kyri and Fioretto, Ferdinando}, year = {2025}, booktitle = {International Conference on Learning Representations}, volume = {13}, doi = {10.48550/arXiv.2410.01786}, } - NAACL 2025Speculative Diffusion Decoding: Accelerating Language Generation through DiffusionJacob K Christopher, Michael Cardei, Brian R Bartoldson, Bhavya Kailkhura, and Ferdinando FiorettoIn Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics, 2025|Abs|Bib|AlphaXivEfficient Natural Language and Speech Processing workshop (NeurIPS 2024)
Speculative decoding has emerged as a widely adopted method to accelerate large language model inference without sacrificing the quality of the model outputs. While this technique has facilitated notable speed improvements by enabling parallel sequence verification, its efficiency remains inherently limited by the reliance on incremental token generation in existing draft models. To overcome this limitation, this paper proposes an adaptation of speculative decoding which uses discrete diffusion models to generate draft sequences. This allows parallelization of both the drafting and verification steps, providing significant speed-ups to the inference process. Our proposed approach, Speculative Diffusion Decoding (SpecDiff), is validated on standard language generation benchmarks and empirically demonstrated to provide a up to 8.7x speed-up over standard generation processes and up to 2.5x speed-up over existing speculative decoding approaches.
@inproceedings{Fioretto:NAACL, title = {Speculative Diffusion Decoding: Accelerating Language Generation through Diffusion}, author = {Christopher, Jacob K and Cardei, Michael and Bartoldson, Brian R and Kailkhura, Bhavya and Fioretto, Ferdinando}, booktitle = {Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics}, year = {2025}, volume = {}, doi = {10.48550/arXiv.2408.05636}, } - AISTATS 2025Differentially Private Data Release on Graphs: Inefficiencies and UnfairnessFerdinando Fioretto, Diptangshu Sen, and Juba Ziani
Networks are crucial components of many sectors, including telecommunications, healthcare, finance, energy, and transportation.The information carried in such networks often contains sensitive user data, like location data for commuters and packet data for online users. Therefore, when considering data release for networks, one must ensure that data release mechanisms do not leak information about individuals, quantified in a precise mathematical sense. Differential Privacy (DP) is the widely accepted, formal, state-of-the-art technique, which has found use in a variety of real-life settings including the 2020 U.S. Census, Apple users’ device data, or Google’s location data. Yet, the use of DP comes with new challenges, as the noise added for privacy introduces inaccuracies or biases and further, DP techniques can also distribute these biases disproportionately across different populations, inducing fairness issues. The goal of this paper is to characterize the impact of DP on bias and unfairness in the context of releasing information about networks, taking a departure from previous work which has studied these effects in the context of private population counts release (such as in the U.S. Census). To this end, we consider a network release problem where the network structure is known to all, but the weights on edges must be released privately. We consider the impact of this private release on a simple downstream decision-making task run by a third-party, which is to find the shortest path between any two pairs of nodes and recommend the best route to users. This setting is of highly practical relevance, mirroring scenarios in transportation networks, where preserving privacy while providing accurate routing information is crucial. Our work provides theoretical foundations and empirical evidence into the bias and unfairness arising due to privacy in these networked decision problems.
@inproceedings{Fioretto:AISTATS25, title = {Differentially Private Data Release on Graphs: Inefficiencies and Unfairness}, author = {Fioretto, Ferdinando and Sen, Diptangshu and Ziani, Juba}, booktitle = {International Conference for Artificial Intelligence and Statistics}, volume = {28}, doi = {10.48550/arXiv.2408.05246}, year = {2025}, } - AAAI 2025Fairness Issues and Mitigations in (Differentially Private) Socio-demographic Data ProcessesOralJoonhyuk Ko, Juba Ziani, Saswat Das, Matt Williams, and Ferdinando Fioretto
Statistical agencies rely on sampling techniques to collect socio-demographic data crucial for policy-making and resource allocation. This paper shows that surveys of important societal relevance introduce sampling errors that unevenly impact group-level estimates, thereby compromising fairness in downstream decisions. To address these issues, this paper introduces an optimization approach modeled on real-world survey design processes, ensuring sampling costs are optimized while maintaining error margins within prescribed tolerances. Additionally, privacy-preserving methods used to determine sampling rates can further impact these fairness issues. The paper explores the impact of differential privacy on the statistics informing the sampling process, revealing a surprising effect: not only the expected negative effect from the addition of noise for differential privacy is negligible, but also this privacy noise can in fact reduce unfairness as it positively biases smaller counts. These findings are validated over an extensive analysis using datasets commonly applied in census statistics.
@inproceedings{Fioretto:AAAI25, title = {Fairness Issues and Mitigations in (Differentially Private) Socio-demographic Data Processes}, author = {Ko, Joonhyuk and Ziani, Juba and Das, Saswat and Williams, Matt and Fioretto, Ferdinando}, year = {2025}, booktitle = {AAAI Conference on Artificial Intelligence}, volume = {39}, doi = {10.48550/arXiv.2408.08471}, } - SatML 2025FairDP: Achieving Fairness Certification with Differential PrivacyKhang Tran, Ferdinando Fioretto, Issa Khalil, My T. Thai, Linh Thi Xuan Phan, and NhatHai PhanIn IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), 2025|Abs|Bib|PDF|Online|AlphaXiv
This paper introduces FairDP, a novel mechanism designed to simultaneously ensure differential privacy (DP) and fairness. FairDP operates by independently training models for distinct individual groups, using group-specific clipping terms to assess and bound the disparate impacts of DP. Throughout the training process, the mechanism progressively integrates knowledge from group models to formulate a comprehensive model that balances privacy, utility, and fairness in downstream tasks. Extensive theoretical and empirical analyses validate the efficacy of FairDP, demonstrating improved trade-offs between model utility, privacy, and fairness compared with existing methods.
@inproceedings{TFKTP:SatML25, author = {Tran, Khang and Fioretto, Ferdinando and Khalil, Issa and Thai, My T. and Phan, Linh Thi Xuan and Phan, NhatHai}, title = {FairDP: Achieving Fairness Certification with Differential Privacy}, booktitle = {IEEE Conference on Secure and Trustworthy Machine Learning (SaTML)}, year = {2025}, doi = {10.1109/SaTML64287.2025.00058}, pages = {956--976}, url = {https://doi.org/10.1109/SaTML64287.2025.00058}, } - AI4D3 2025Constrained Molecular Generation with Discrete Diffusion for Drug DiscoveryBest Paper AwardMichael Cardei, Jacob K Christopher, Thomas Hartvigsen, Bhavya Kailkhura, and Ferdinando FiorettoIn NeurIPS 2025 Workshop on AI Virtual Cells and Instruments: A New Era in Drug Discovery and Development, 2025|Abs|Bib|Online
Discrete diffusion models are a class of generative models that construct sequences by progressively denoising samples from a categorical noise distribution. In life science setting, such as molecular strings (SMILES) and other biological sequence design settings, these models have emerged as a promising alternative to autoregressive architectures, presenting an opportunity to enforce sequence-level constraints, a capability that existing left-to-right sequence design cannot natively provide. This paper capitalizes on this opportunity by introducing (CDD), a novel integration of differentiable constraint optimization within the diffusion process to maintain policy, safety, and design-property adherence throughout generation. Unlike conventional generators that often rely on post-hoc filtering or model retraining for controllable generation, CDD directly imposes constraints into the discrete diffusion sampling process, resulting in a training-free and effective virtual-instrument approach. Experiments in property adherence molecular design, toxicity-bounded generation, and novelty enforcement demonstrate that CDD achieves in a diverse array of tasks outperforming auto-regressive and existing discrete diffusion approaches. CDD is a practical virtual instrument for molecular design and lays the groundwork for discrete-diffusion-based virtual-cell workflows in drug discovery and development.
@inproceedings{Fioretto:AI4D3, title = {Constrained Molecular Generation with Discrete Diffusion for Drug Discovery}, author = {Cardei, Michael and Christopher, Jacob K and Hartvigsen, Thomas and Kailkhura, Bhavya and Fioretto, Ferdinando}, booktitle = {NeurIPS 2025 Workshop on AI Virtual Cells and Instruments: A New Era in Drug Discovery and Development}, year = {2025}, volume = {38}, pages = {}, } - AI4Mat 2025Constrained Diffusion for Accelerated Structure Relaxation of Inorganic Solids with Point DefectsJingyi Cui, Jacob K Christopher, Ankita Biswas, Prasanna V. Balachandran, and Ferdinando Fioretto
Point defects affect material properties by altering electronic states and modifying local bonding environments. However, high-throughput first-principles simulations of point defects are costly due to large simulation cells and complex energy landscapes. To this end, we propose a generative framework for simulating point defects, overcoming the limits of costly first-principles simulators. By leveraging a primal-dual algorithm, we introduce a constraint-aware diffusion model which outperforms existing constrained diffusion approaches in this domain. Across six defect configuration settings for Bi_2 Te_3, the proposed approach provides state-of-the-art performance generating physically grounded structures.
@inproceedings{Fioretto:AI4Mat25, title = {Constrained Diffusion for Accelerated Structure Relaxation of Inorganic Solids with Point Defects}, author = {Cui, Jingyi and Christopher, Jacob K and Biswas, Ankita and Balachandran, Prasanna V. and Fioretto, Ferdinando}, booktitle = {AI for Accelerated Materials Design - NeurIPS}, volume = {1}, year = {2025}, doi = {}, } - CoLoRAI 2025Low-rank finetuning for LLMs: A fairness perspectiveBest paper awardSaswat Das, Marco Romanelli, Cuong Tran, Bhavya Kailkhura, and Ferdinando Fioretto
Low-rank approximation techniques have become the de facto standard for fine-tuning Large Language Models (LLMs) due to their reduced computational and memory requirements. This paper investigates the effectiveness of these methods in capturing the shift of fine-tuning datasets from the initial pre-trained data distribution. Our findings reveal that there are cases in which low-rank fine-tuning falls short in learning such shifts. This, in turn, produces non-negligible side effects, especially when fine-tuning is adopted for toxicity mitigation in pre-trained models, or in scenarios where it is important to provide fair models. Through comprehensive empirical evidence on several models, datasets, and tasks, we show that low-rank fine-tuning inadvertently preserves undesirable biases and toxic behaviors. We also show that this extends to sequential decision-making tasks, emphasizing the need for careful evaluation to promote responsible LLMs development.
@inproceedings{Fioretto:colorai25, title = {Low-rank finetuning for LLMs: A fairness perspective}, author = {Das, Saswat and Romanelli, Marco and Tran, Cuong and Kailkhura, Bhavya and Fioretto, Ferdinando}, year = {2025}, booktitle = {AAAI CoLoRAI Workshop}, doi = {10.48550/arXiv.2405.18572}, } - MAPF 2025Multi-Agent Path Finding in Continuous Spaces with Projected Diffusion ModelsOralJinhao Liang, Jacob Christopher, Sven Koenig, and Ferdinando FiorettoCombining AI and OR/MS for Better Trustworthy Decision Making (AAAI 2025)
Multi-Agent Path Finding (MAPF) is a fundamental problem in robotics, requiring the computation of collision-free paths for multiple agents moving from their respective start to goal positions. The complexity of coordinating multiple agents in a shared environment poses significant challenges, especially in continuous spaces where traditional optimization algorithms fall short with scalability. Recently, diffusion models have been successfully applied to single-agent contexts, offering advantages such as modeling complex trajectory distributions and generating smooth paths that navigate high-dimensional spaces effectively. However, such models have struggled to ensure constraints feasibility, such as non-collision. Additionally, extending diffusion models to MAPF brings an additional level of complexity due to the necessity to satisfy inter-agent constraints. In this work, we propose a novel combination of constrained optimization with diffusion models for MAPF in continuous spaces. Such a unique combination enables the direct generation of feasible multi-agent trajectories, considering collision avoidance and kinematic feasibility. Additionally, the paper proposes an augmented Lagrangian method that reduces complexity and enhances practicality for real-time applications. The effectiveness of our approach is demonstrated across various simulated scenarios.
@inproceedings{Fioretto:MAPF25, title = {Multi-Agent Path Finding in Continuous Spaces with Projected Diffusion Models}, author = {Liang, Jinhao and Christopher, Jacob and Koenig, Sven and Fioretto, Ferdinando}, year = {2025}, volume = {39}, booktitle = {The 6th International Workshop on Multi-Agent Path Finding, at AAAI}, } - AIORMS 2025OPF-Net: Real-Time Stability Constrained AC Optimal Power FlowVincenzo Di Vito, Mostafa Mohammadian, Kyri Baker, and Ferdinando FiorettoIn AAAI 2025 Bridge on Explainable AI, Energy and Critical Infrastructure Systems, 2025|Abs|Bib|AlphaXiv
Recent developments in applying machine learning to address Alternating Current Optimal Power Flow (AC OPF) problems have demonstrated significant potential in providing close to optimal solutions for generator dispatch in near real-time. While these learning to optimize methods have demonstrated remarkable performance on steady-state operations, practical applications often demand compliance with dynamic constraints when used for fast-timescale optimization. This paper addresses this gap and develops a real-time stability-constrained OPF model (DynOPF-Net) that simultaneously addresses both optimality and dynamical stability within learning-assisted grid operations. The model is a unique integration of learning to optimize that learns a mapping from load conditions to OPF solutions, capturing the OPF’s physical and engineering constraints, with Neural Ordinary Differential Equations, capturing generator dynamics, enabling the inclusion of a subset of stability constraints. Numerical results on the WSCC 9-bus and IEEE 57-bus benchmark systems demonstrate that DynOPF-Net can produce highly accurate AC-OPF solutions while also ensuring system stability, contrasting the unstable results obtained by state-of-the-art LtO methods.
@inproceedings{Fioretto:AIAIbridge25b, title = {OPF-Net: Real-Time Stability Constrained AC Optimal Power Flow}, author = {Vito, Vincenzo Di and Mohammadian, Mostafa and Baker, Kyri and Fioretto, Ferdinando}, booktitle = {AAAI 2025 Bridge on Explainable AI, Energy and Critical Infrastructure Systems}, volume = {1}, year = {2025}, doi = {}, }
2024
- preprint 2024End-to-End Optimization and Learning of Fair Court SchedulesMy H. Dinh, James Kotary, Lauryn P. Gouldin, William Yeoh, and Ferdinando Fioretto
Criminal courts across the United States handle millions of cases every year, and the scheduling of those cases must accommodate a diverse set of constraints, including the preferences and availability of courts, prosecutors, and defense teams. When criminal court schedules are formed, defendants’ scheduling preferences often take the least priority, although defendants may face significant consequences (including arrest or detention) for missed court dates. Additionally, studies indicate that defendants’ nonappearances impose costs on the courts and other system stakeholders. To address these issues, courts and commentators have begun to recognize that pretrial outcomes for defendants and for the system would be improved with greater attention to court processes, including court scheduling practices. There is thus a need for fair criminal court pretrial scheduling systems that account for defendants’ preferences and availability, but the collection of such data poses logistical challenges. Furthermore, optimizing schedules fairly across various parties’ preferences is a complex optimization problem, even when such data is available. In an effort to construct such a fair scheduling system under data uncertainty, this paper proposes a joint optimization and learning framework that combines machine learning models trained end-to-end with efficient matching algorithms. This framework aims to produce court scheduling schedules that optimize a principled measure of fairness, balancing the availability and preferences of all parties.
@article{Fioretto:24j, title = {End-to-End Optimization and Learning of Fair Court Schedules}, author = {Dinh, My H. and Kotary, James and Gouldin, Lauryn P. and Yeoh, William and Fioretto, Ferdinando}, journal = {CoRR}, volume = {abs/2410.17415}, doi = {10.48550/arXiv.2410.17415}, year = {2024}, } - preprint 2024Learning Constrained Optimization with Deep Augmented Lagrangian MethodsJames Kotary, and Ferdinando Fioretto
Learning to Optimize (LtO) is a problem setting in which a machine learning (ML) model is trained to emulate a constrained optimization solver. Learning to produce optimal and feasible solutions subject to complex constraints is a difficult task, but is often made possible by restricting the input space to a limited distribution of related problems. Most LtO methods focus on directly learning solutions to the primal problem, and applying correction schemes or loss function penalties to encourage feasibility. This paper proposes an alternative approach, in which the ML model is trained instead to predict dual solution estimates directly, from which primal estimates are constructed to form dual-feasible solution pairs. This enables an end-to-end training scheme is which the dual objective is maximized as a loss function, and solution estimates iterate toward primal feasibility, emulating a Dual Ascent method. First it is shown that the poor convergence properties of classical Dual Ascent are reflected in poor convergence of the proposed training scheme. Then, by incorporating techniques from practical Augmented Lagrangian methods, we show how the training scheme can be improved to learn highly accurate constrained optimization solvers, for both convex and nonconvex problems.
@article{Fioretto:24c, title = {Learning Constrained Optimization with Deep Augmented Lagrangian Methods}, author = {Kotary, James and Fioretto, Ferdinando}, journal = {CoRR}, volume = {abs/2403.03454}, doi = {10.48550/arXiv.2403.03454}, year = {2024}, } - preprint 2024Analyzing and Enhancing the Backward-Pass Convergence of Unrolled OptimizationJames Kotary, Jacob Christopher, My H Dinh, and Ferdinando Fioretto
The integration of constrained optimization models as components in deep networks has led to promising advances on many specialized learning tasks. A central challenge in this setting is backpropagation through the solution of an optimization problem, which often lacks a closed form. One typical strategy is algorithm unrolling, which relies on automatic differentiation through the entire chain of operations executed by an iterative optimization solver. This paper provides theoretical insights into the backward pass of unrolled optimization, showing that it is asymptotically equivalent to the solution of a linear system by a particular iterative method. Several practical pitfalls of unrolling are demonstrated in light of these insights, and a system called Folded Optimization is proposed to construct more efficient backpropagation rules from unrolled solver implementations. Experiments over various end-to-end optimization and learning tasks demonstrate the advantages of this system both computationally, and in terms of flexibility over various optimization problem forms.
@article{Fioretto:24a, title = {Analyzing and Enhancing the Backward-Pass Convergence of Unrolled Optimization}, author = {Kotary, James and Christopher, Jacob and Dinh, My H and Fioretto, Ferdinando}, journal = {CoRR}, volume = {abs/2312.17394}, doi = {10.48550/arXiv.2312.17394}, year = {2024}, } - JAIR 2024Decision-Focused Learning: Foundations, State of the Art, Benchmark and Future OpportunitiesJayanta Mandi, James Kotary, Senne Berden, Maxime Mulamba, Victor Bucarey, Tias Guns, and Ferdinando Fioretto
Decision-focused learning (DFL) is an emerging paradigm in machine learning which trains a model to optimize decisions, integrating prediction and optimization in an end-to-end system. This paradigm holds the promise to revolutionize decision-making in many real-world applications which operate under uncertainty, where the estimation of unknown parameters within these decision models often becomes a substantial roadblock. This paper presents a comprehensive review of DFL. It provides an in-depth analysis of the various techniques devised to integrate machine learning and optimization models, introduces a taxonomy of DFL methods distinguished by their unique characteristics, and conducts an extensive empirical evaluation of these methods proposing suitable benchmark dataset and tasks for DFL. Finally, the study provides valuable insights into current and potential future avenues in DFL research.
@article{Fioretto:jair24, author = {Mandi, Jayanta and Kotary, James and Berden, Senne and Mulamba, Maxime and Bucarey, Victor and Guns, Tias and Fioretto, Ferdinando}, title = {Decision-Focused Learning: Foundations, State of the Art, Benchmark and Future Opportunities}, journal = {Journal of Artificial Intelligence Research}, volume = {81}, pages = {1623--1701}, year = {2024}, doi = {10.48550/arXiv.2307.13565}, } - NeurIPS 2024Constrained Synthesis with Projected Diffusion ModelsJacob K Christopher, Stephen Baek, and Ferdinando FiorettoAlso appeared in: Machine Learning and the Physical Sciences Workshop (NeurIPS 2024)
Generative diffusion models excel at robustly synthesizing coherent content from raw noise through a sequential process. However, their direct application in scenarios requiring outputs to adhere to specific, stringent criteria faces several severe challenges. This paper aims at overcome these challenges and introduces Projected Generative Diffusion Models (PGDM), an approach that recast traditional diffusion models sampling into a constrained-optimization problem. This enables the application of an iterative projections method to ensure that generated data faithfully adheres to specified constraints or physical principles. This paper provides theoretical support for the ability of PGDM to synthesize outputs from a feasible subdistribution under a restricted class of constraints while also providing large empirical evidence in the case of complex non-convex constraints and ordinary differential equations. These capabilities are demonstrated by physics-informed motion in video generation, trajectory optimization in path planning, and morphometric properties adherence in material science.
@inproceedings{Fioretto:NeurIPS24, title = {Constrained Synthesis with Projected Diffusion Models}, author = {Christopher, Jacob K and Baek, Stephen and Fioretto, Ferdinando}, booktitle = {Advances in Neural Information Processing Systems}, year = {2024}, volume = {37}, pages = {}, publisher = {Curran Associates, Inc.}, } - NeurIPS 2024Physics-Aware Generative Diffusion Models for Micro-structure Material DesignSpotlightJacob K Christopher, Stephen Baek, and Ferdinando Fioretto
Generative diffusion models excel at robustly synthesizing coherent content from raw noise through a sequential process. However, their direct application in scenarios requiring outputs to adhere to specific, stringent criteria faces several severe challenges. This paper aims at overcome these challenges and introduces Projected Generative Diffusion Models (PGDM), an approach that recast traditional diffusion models sampling into a constrained-optimization problem. This enables the application of an iterative projections method to ensure that generated data faithfully adheres to specified constraints or physical principles. This paper provides theoretical support for the ability of PGDM to synthesize outputs from a feasible subdistribution under a restricted class of constraints while also providing large empirical evidence in the case of complex non-convex constraints and ordinary differential equations. These capabilities are demonstrated by physics-informed motion in video generation, trajectory optimization in path planning, and morphometric properties adherence in material science.
@article{Fioretto:NeurIPS24b, title = {Physics-Aware Generative Diffusion Models for Micro-structure Material Design}, author = {Christopher, Jacob K and Baek, Stephen and Fioretto, Ferdinando}, journal = {AI 4 Material Science workshop, at NeurIPS}, year = {2024}, volume = {37}, pages = {}, url = {https://neurips.cc/virtual/2024/100112}, } - NeurIPS 2024The Data Minimization Principle in Machine LearningOralFerdinando Fioretto, Prakhar Ganesh, Cuong Tran, and Reza ShokriWorkshop on Generative AI and Law (ICML 2024)
The principle of data minimization requires organizations to collect, process, and retain only personal data that is adequate, relevant, and limited to what is necessary for specified objectives. It’s grounded in the expectation that not all collected data is essential for the objective and, instead, contributes to a heightened risk of information leakage. However, despite its legal significance and endorsement by global data protection regulations, the data minimization principle lacks a mathematical formalization suitable for real-world ML applications. In particular, the current discourse on data minimization practices often overlooks two crucial aspects: \bf (1) the individualized nature of minimization (e.g., information that is unimportant for an individual may be critical for another) and \bf (2) its intrinsic link to data privacy. To overcome these limitations, this paper introduces a formal framework for data minimization in ML while being faithful to its legal notion; adapts and evaluates various optimization algorithms to solve the problem of data minimization; and analyzes their compatibility with real-world privacy. In particular, we seek to answer a critical question: “Do data minimization requirements in various regulations meet privacy expectations in legal frameworks?” Our evaluations reveal that the answer is, unfortunately, no. While being an implicit intention, the requirements of data minimization are not necessarily aligned with risk of reconstruction and re-identification and thus may not provide the expected privacy protection. With this work, we aim to lay down a path for future research for developing ML systems that comply with the legal requirements of data minimization.
@inproceedings{Fioretto:neurIPS_w24a, title = {The Data Minimization Principle in Machine Learning}, author = {Fioretto, Ferdinando and Ganesh, Prakhar and Tran, Cuong and Shokri, Reza}, booktitle = {NeurIPS 2024 Workshop on Regulatory ML}, year = {2024}, } - CDC 2024Metric Learning to Accelerate Convergence of Operator Splitting Methods for Differentiable Parametric ProgrammingEthan King, James Kotary, Ferdinando Fioretto, and Jan Drgona
Recent work has shown a variety of ways in which machine learning can be used to accelerate the solution of constrained optimization problems. Increasing demand for real-time decision-making capabilities in applications such as artificial intelligence and optimal control has led to a variety of approaches, based on distinct strategies. This work proposes a novel approach to learning optimization, in which the underlying metric space of a proximal operator splitting algorithm is learned so as to maximize its convergence rate. While prior works in optimization theory have derived optimal metrics for limited classes of problems, the results do not extend to many practical problem forms including general Quadratic Programming (QP). This paper shows how differentiable optimization can enable the end-to-end learning of proximal metrics, enhancing the convergence of proximal algorithms for QP problems beyond what is possible based on known theory. Additionally, the results illustrate a strong connection between the learned proximal metrics and active constraints at the optima, leading to an interpretation in which the learning of proximal metrics can be viewed as a form of active set learning.
@inproceedings{Fioretto:CCD24, title = {Metric Learning to Accelerate Convergence of Operator Splitting Methods for Differentiable Parametric Programming}, author = {King, Ethan and Kotary, James and Fioretto, Ferdinando and Drgona, Jan}, booktitle = {IEEE Conference on Decision and Control (CDC)}, year = {2024}, doi = {10.48550/arXiv.2404.00882}, } - ECAI 2024Learning Joint Models of Prediction and OptimizationJames Kotary, Vincenzo Di Vito, Jacob Christopher, Pascal Van Hentenryck, and Ferdinando Fioretto
The Predict-Then-Optimize framework uses machine learning models to predict unknown parameters of an optimization problem from features before solving. While this traditional setting is common to many real-world decision processes, recently it has been shown that decision quality can be substantially improved by solving and differentiating the optimization problem in the training loop. However, this approach requires solving optimization in the loop along with handcrafted, problem-specific rules for backpropagation through the optimization step, challenging its applicability to a broad class of optimization problems. This paper proposes an alternative method, in which optimal solutions are learned directly from the observable features by joint predictive models. The approach is generic, and based on an adaptation of the Learning-to-Optimize paradigm, from which a rich variety of existing techniques can be employed. Experimental evaluations show the ability of several Learning-to-Optimize methods to provide efficient and accurate solutions to an array of challenging Predict-Then-Optimize problems
@inproceedings{Fioretto:ecai24, title = {Learning Joint Models of Prediction and Optimization}, author = {Kotary, James and {Di Vito}, Vincenzo and Christopher, Jacob and {Van Hentenryck}, Pascal and Fioretto, Ferdinando}, booktitle = {European Conference on Artificial Intelligence}, publisher = {IOS Press}, pages = {2476--2483}, year = {2024}, doi = {10.3233/FAIA240775}, url = {https://doi.org/10.3233/FAIA240775}, } - ICML 2024On The Fairness Impacts of Hardware Selection in Machine LearningSree Harsha Nelaturu, Nishaanth Kanna Ravichandran, Cuong Tran, Sara Hooker, and Ferdinando Fioretto
In the machine learning ecosystem, hardware selection is often regarded as a mere utility, overshadowed by the spotlight on algorithms and data. This oversight is particularly problematic in contexts like ML-as-a-service platforms, where users often lack control over the hardware used for model deployment. How does the choice of hardware impact generalization properties? This paper investigates the influence of hardware on the delicate balance between model performance and fairness. We demonstrate that hardware choices can exacerbate existing disparities, attributing these discrepancies to variations in gradient flows and loss surfaces across different demographic groups. Through both theoretical and empirical analysis, the paper not only identifies the underlying factors but also proposes an effective strategy for mitigating hardware-induced performance imbalances.
@inproceedings{Fioretto:icml24a, title = {On The Fairness Impacts of Hardware Selection in Machine Learning}, author = {Nelaturu, Sree Harsha and Ravichandran, Nishaanth Kanna and Tran, Cuong and Hooker, Sara and Fioretto, Ferdinando}, booktitle = {International Conference on Machine Learning}, publisher = {}, doi = {10.48550/arXiv.2312.03886}, year = {2024}, } - ICML 2024Disparate Impact on Group Accuracy of Linearization for Private InferenceSaswat Das, Marco Romanelli, and Ferdinando Fioretto
Ensuring privacy-preserving inference on cryptographically secure data is a well-known computational challenge. To alleviate the bottleneck of costly cryptographic computations in non-linear activations, recent methods have suggested linearizing a targeted portion of these activations in neural networks. This technique results in significantly reduced runtimes with often negligible impacts on accuracy. In this paper, we demonstrate that such computational benefits may lead to increased fairness costs. Specifically, we find that reducing the number of ReLU activations disproportionately decreases the accuracy for minority groups compared to majority groups. To explain these observations, we provide a mathematical interpretation under restricted assumptions about the nature of the decision boundary, while also showing the prevalence of this problem across widely used datasets and architectures. Finally, we show how a simple procedure altering the fine-tuning step for linearized models can serve as an effective mitigation strategy.
@inproceedings{Fioretto:icml24b, title = {Disparate Impact on Group Accuracy of Linearization for Private Inference}, author = {Das, Saswat and Romanelli, Marco and Fioretto, Ferdinando}, booktitle = {International Conference on Machine Learning}, publisher = {}, doi = {10.48550/arXiv.2402.03629}, year = {2024}, } - ICML 2024Differentiable Approximations of Fair OWA OptimizationMy H. Dinh, James Kotary, and Ferdinando Fioretto
Decision processes in AI and operations research often involve parametric optimization problems, whose unknown parameters must be inferred from data. The Predict-Then-Optimize (PtO) paradigm maximizes decision quality by training parametric prediction models end-to-end with subsequent constrained optimization. This paper extends PtO to handle the optimization of nondifferentiable Ordered Weighted Averaging (OWA) objectives, known for their ability to ensure fair and robust solutions with respect to multiple objectives. By proposing efficient differentiable approximations of OWA optimization, it provides a framework for integrating fair optimization concepts with parametric prediction under uncertainty.
@inproceedings{Fioretto:icml_w24b, title = {Differentiable Approximations of Fair OWA Optimization}, author = {Dinh, My H. and Kotary, James and Fioretto, Ferdinando}, booktitle = {ICML 2024 Workshop on Differentiable Almost Everything}, url = {https://openreview.net/forum?id=NBt4ZBOFth}, year = {2024}, } - UAI 2024End-to-End Learning for Fair Multiobjective Optimization Under UncertaintyMy H. Dinh, James Kotary, and Ferdinando Fioretto
Many decision processes in artificial intelligence and operations research are modeled by parametric optimization problems whose defining parameters are unknown and must be inferred from observable data. The Predict-Then-Optimize (PtO) paradigm in machine learning aims to maximize downstream decision quality by training the parametric inference model end-to-end with the subsequent constrained optimization. This requires backpropagation through the optimization problem using approximation techniques specific to the problem’s form, especially for nondifferentiable linear and mixed-integer programs. This paper extends the PtO methodology to optimization problems with nondifferentiable Ordered Weighted Averaging (OWA) objectives, known for their ability to ensure properties of fairness and robustness in decision models. Through a collection of training techniques and proposed application settings, it shows how optimization of OWA functions can be effectively integrated with parametric prediction for fair and robust optimization under uncertainty.
@inproceedings{Fioretto:uai24, title = {End-to-End Learning for Fair Multiobjective Optimization Under Uncertainty}, author = {Dinh, My H. and Kotary, James and Fioretto, Ferdinando}, booktitle = {Conference on Uncertainty in Artificial Intelligence}, publisher = {}, doi = {10.48550/arXiv.2402.07772}, year = {2024}, } - IJCAI 2024On the Effects of Fairness to Adversarial VulnerabilityCuong Tran, Keyu Zhu, Pascal Van Hentenryck, and Ferdinando Fioretto
The remarkable performance of deep learning models and their applications in consequential domains (e.g., facial recognition) introduces important challenges at the intersection of equity and security. Fairness and robustness are two desired notions often required in learning models. Fairness ensures that models do not disproportionately harm (or benefit) some groups over others, while robustness measures the models’ resilience against small input perturbations. This paper shows the existence of a dichotomy between fairness and robustness, and analyzes when achieving fairness decreases the model robustness to adversarial samples. The reported analysis sheds light on the factors causing such contrasting behavior, suggesting that distance to the decision boundary across groups as a key explainer for this behavior. Extensive experiments on non-linear models and different architectures validate the theoretical findings in multiple vision domains. Finally, the paper proposes a simple, yet effective, solution to construct models achieving good tradeoffs between fairness and robustness.
@inproceedings{Fioretto:ijcai24, author = {Tran, Cuong and Zhu, Keyu and {Van Hentenryck}, Pascal and Fioretto, Ferdinando}, title = {On the Effects of Fairness to Adversarial Vulnerability}, booktitle = {International Joint Conference on Artificial Intelligence}, publisher = {ijcai.org}, pages = {521--529}, year = {2024}, doi = {10.24963/ijcai.2024/58}, url = {https://www.ijcai.org/proceedings/2024/58}, } - AAAI 2024Finding epsilon and delta of Traditional Disclosure Control SystemsSaswat Das, Keyu Zhu, Christine Task, Pascal Van Hentenryck, and Ferdinando Fioretto
This paper analyzes the privacy of traditional Statistical Disclosure Control (SDC) systems under a differential privacy interpretation. SDCs, such as cell suppression and swapping promise to safeguard the confidentiality of data and are routinely adopted in data analyses with profound societal and economic impacts. Through a formal analysis and empirical evaluation on US Census data, the paper shows that widely adopted SDC systems not only induce vastly larger privacy losses than classical differential privacy mechanisms, but, they may also come at a cost of larger accuracy and fairness.
@inproceedings{Fioretto:aaai24, author = {Das, Saswat and Zhu, Keyu and Task, Christine and Van Hentenryck, Pascal and Fioretto, Ferdinando}, title = {Finding epsilon and delta of Traditional Disclosure Control Systems}, booktitle = {AAAI Conference on Artificial Intelligence}, publisher = {{AAAI} Press}, volume = {38}, pages = {22013--22020}, year = {2024}, url = {https://ojs.aaai.org/index.php/AAAI/article/view/30204}, doi = {10.1609/aaai.v38i20.30204}, number = {20} } - FAccT 2024Learning Fair Ranking Policies via Differentiable Optimization of Ordered Weighted AveragesMy H. Dinh, James Kotary, and Ferdinando FiorettoIn ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT), 2024|Abs|Bib|Online|AlphaXiv
Learning to Rank (LTR) is one of the most widely used machine learning applications. It is a key component in platforms with profound societal impacts, including job search, healthcare information retrieval, and social media content feeds. Conventional LTR models have been shown to produce biases results, stimulating a discourse on how to address the disparities introduced by ranking systems that solely prioritize user relevance. However, while several models of fair learning to rank have been proposed, they suffer from deficiencies either in accuracy or efficiency, thus limiting their applicability to real-world ranking platforms. This paper shows how efficiently-solvable fair ranking models, based on the optimization of Ordered Weighted Average (OWA) functions, can be integrated into the training loop of an LTR model to achieve favorable balances between fairness, user utility, and runtime efficiency. In particular, this paper is the first to show how to backpropagate through constrained optimizations of OWA objectives, enabling their use in integrated prediction and decision models.
@inproceedings{Fioretto:facct24a, title = {Learning Fair Ranking Policies via Differentiable Optimization of Ordered Weighted Averages}, author = {Dinh, My H. and Kotary, James and Fioretto, Ferdinando}, booktitle = {ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT)}, year = {2024}, pages = {2508--2517}, doi = {10.1145/3630106.3661932}, }
2023
- AFCP 2023Awa Dieng, Miriam Rateike, Golnoosh Farnadi, Ferdinando Fioretto, Matt Kusner, and Jessica SchrouffIn Proceedings of the Workshop on Algorithmic Fairness through the Lens of Causality and Privacy, 2023Workshop report|Bib|PDF|Online
@inproceedings{Fioretto:AFCP23, title = {Algorithmic Fairness through the Lens of Causality and Privacy (AFCP) 2022}, author = {Dieng, Awa and Rateike, Miriam and Farnadi, Golnoosh and Fioretto, Ferdinando and Kusner, Matt and Schrouff, Jessica}, booktitle = {Proceedings of the Workshop on Algorithmic Fairness through the Lens of Causality and Privacy}, pages = {1--6}, year = {2023}, volume = {214}, series = {Proceedings of Machine Learning Research}, publisher = {PMLR}, note = {Workshop report} } - ArXiv 2023Context-Aware Differential Privacy for Language ModelingMy H. Dinh, and Ferdinando Fioretto
The remarkable ability of language models (LMs) has also brought challenges at the interface of AI and security. A critical challenge pertains to how much information these models retain and leak about the training data. This is particularly urgent as the typical development of LMs relies on huge, often highly sensitive data, such as emails and chat logs. To contrast this shortcoming, this paper introduces Context-Aware Differentially Private Language Model (CADP-LM) , a privacy-preserving LM framework that relies on two key insights: First, it utilizes the notion of context to define and audit the potentially sensitive information. Second, it adopts the notion of Differential Privacy to protect sensitive information and characterize the privacy leakage. A unique characteristic of CADP-LM is its ability to target the protection of sensitive sentences and contexts only, providing a highly accurate private model. Experiments on a variety of datasets and settings demonstrate these strengths of CADP-LM.
@article{DF:23, author = {Dinh, My H. and Fioretto, Ferdinando}, title = {Context-Aware Differential Privacy for Language Modeling}, journal = {CoRR}, volume = {abs/2301.12288}, year = {2023}, doi = {10.48550/arXiv.2301.12288}, } - EPSR 2023Gradient-enhanced physics-informed neural networks for power systems operational supportMostafa Mohammadian, Kyri Baker, and Ferdinando Fioretto
The application of deep learning methods to speed up the challenging power system problems has recently shown very encouraging results. However, power system dynamics are not snapshot, steady-state operations. These dynamics must be considered to ensure that the optimal solutions provided by these models adhere to practical constraints to avoid frequency fluctuations and grid instabilities. Unfortunately, dynamic system models based on ordinary or partial differential equations are frequently unsuitable for direct application in control or state estimates due to their high computational costs. To address these challenges, this paper introduces a machine learning method to approximate the behavior of power systems dynamics in near real-time. The proposed framework is based on gradient-enhanced physics-informed neural networks (gPINNs) and encodes the underlying physical laws governing power systems. A key characteristic of the proposed gPINN is its ability to train without the need of generating expensive training data. The paper illustrates the potential of the proposed approach in both forward and inverse problems in a single-machine infinite bus system and a three-bus power network for predicting rotor angles and frequency, and uncertain parameters such as inertia and damping to showcase its potential for a range of power systems applications. The model exhibited high accuracy in predicting the variables, achieving a range of 0.533–4.092 and an average L2 relative error improvement of up to 13.30× compared to the PINN model. The computational performance of the proposed gPINN model was compared to a conventional solver, revealing a remarkable speed-up of 31 to 171 times faster in solving differential–algebraic systems of equations in power systems.
@article{MBF:epsr23, author = {Mohammadian, Mostafa and Baker, Kyri and Fioretto, Ferdinando}, journal = {Electric Power Systems Research}, volume = {223}, pages = {109551}, year = {2023}, issn = {0378-7796}, doi = {10.1016/j.epsr.2023.109551}, url = {https://doi.org/10.1016/j.epsr.2023.109551}, } - NeurIPS 2023Data Minimization at Inference TimeCuong Tran, and Ferdinando Fioretto
In domains with high stakes such as law, recruitment, and healthcare, learning models frequently rely on sensitive user data for inference, necessitating the complete set of features. This not only poses significant privacy risks for individuals but also demands substantial human effort from organizations to verify information accuracy. This paper asks whether it is necessary to use all input features for accurate predictions at inference time. The paper demonstrates that, in a personalized setting, individuals may only need to disclose a small subset of their features without compromising decision-making accuracy. The paper also provides an efficient sequential algorithm to determine the appropriate attributes for each individual to provide. Evaluations across various learning tasks show that individuals can potentially report as little as 10% of their information while maintaining the same accuracy level as a model that employs the full set of user information.
@inproceedings{TF:neurips23, author = {Tran, Cuong and Fioretto, Ferdinando}, title = {Data Minimization at Inference Time}, booktitle = {Advances in Neural Information Processing Systems}, year = {2023}, volume = {36}, pages = {}, publisher = {Curran Associates, Inc.}, } - NeurIPS 2023Price-Aware Deep Learning for Electricity MarketsVladimir Dvorkin, and Ferdinando FiorettoIn NeurIPS 2023 Workshop: Tackling Climate Change with Machine Learning, 2023|Abs|Bib|Online|AlphaXiv
While deep learning gradually penetrates operational planning, its inherent prediction errors may significantly affect electricity prices. This letter examines how prediction errors propagate into electricity prices, revealing notable pricing errors and their spatial disparity in congested power systems. To improve fairness, we propose to embed electricity market-clearing optimization as a deep learning layer. Differentiating through this layer allows for balancing between prediction and pricing errors, as oppose to minimizing prediction errors alone. This layer implicitly optimizes fairness and controls the spatial distribution of price errors across the system. We showcase the price-aware deep learning in the nexus of wind power forecasting and short-term electricity market clearing.
@inproceedings{DF:ccai23, author = {Dvorkin, Vladimir and Fioretto, Ferdinando}, title = {Price-Aware Deep Learning for Electricity Markets}, booktitle = {NeurIPS 2023 Workshop: Tackling Climate Change with Machine Learning}, publisher = {}, pages = {}, year = {2023}, url = {https://www.climatechange.ai/events/neurips2023}, doi = {10.48550/arXiv.2308.01436}, } - IJCAI 2023SF-PATE: Scalable, Fair, and Private Aggregation of Teacher EnsemblesCuong Tran, Keyu Zhu, Ferdinando Fioretto, and Pascal Van Hentenryck
A critical concern in data-driven processes is to build models whose outcomes do not discriminate against some demographic groups, including gender, ethnicity, or age. To ensure non-discrimination in learning tasks, knowledge of the group attributes is essential. However, in practice, these attributes may not be available due to legal and ethical requirements. To address this challenge, this paper studies a model that protects the privacy of the individuals’ sensitive information while also allowing it to learn non-discriminatory predictors. A key characteristic of the proposed model is to enable the adoption of off-the-selves and non-private fair models to create a privacy-preserving and fair model. The paper analyzes the relation between accuracy, privacy, and fairness, and the experimental evaluation illustrates the benefits of the proposed models on several prediction tasks. In particular, this proposal is the first to allow both scalable and accurate training of private and fair models for very large neural networks.
@inproceedings{TZFvH:ijcai23, author = {Tran, Cuong and Zhu, Keyu and Fioretto, Ferdinando and Hentenryck, Pascal Van}, title = {{SF-PATE:} Scalable, Fair, and Private Aggregation of Teacher Ensembles}, booktitle = {International Joint Conference on Artificial Intelligence}, publisher = {ijcai.org}, pages = {501--509}, year = {2023}, url = {https://doi.org/10.24963/ijcai.2023/56}, doi = {10.24963/ijcai.2023/56}, } - IJCAI 2023On the Fairness Impacts of Private Ensembles ModelsCuong Tran, and Ferdinando FiorettoAlso appeared in: PPAI (AAAI 2022)
The Private Aggregation of Teacher Ensembles (PATE) is a machine learning framework that enables the creation of private models through the combination of multiple "teacher" models and a "student" model. The student model learns to predict an output based on the voting of the teachers, and the resulting model satisfies differential privacy. PATE has been shown to be effective in creating private models in semi-supervised settings or when protecting data labels is a priority. This paper explores whether the use of PATE can result in unfairness, and demonstrates that it can lead to accuracy disparities among groups of individuals. The paper also analyzes the algorithmic and data properties that contribute to these disproportionate impacts, why these aspects are affecting different groups disproportionately, and offers recommendations for mitigating these effects.
@inproceedings{TF:ijcai23, author = {Tran, Cuong and Fioretto, Ferdinando}, title = {On the Fairness Impacts of Private Ensembles Models}, booktitle = {International Joint Conference on Artificial Intelligence}, publisher = {ijcai.org}, pages = {510--518}, year = {2023}, url = {https://doi.org/10.24963/ijcai.2023/57}, doi = {10.24963/ijcai.2023/57}, } - IJCAI 2023Differentiable Model Selection for Ensemble LearningJames Kotary, Vincenzo Di Vito, and Ferdinando Fioretto
Model selection is a strategy aimed at creating accurate and robust models by identifying the optimal model for classifying any particular input sample. This paper proposes a novel framework for differentiable selection of groups of models by integrating machine learning and combinatorial optimization. The framework is tailored for ensemble learning with a strategy that learns to combine the predictions of appropriately selected pre-trained ensemble models. It does so by modeling the ensemble learning task as a differentiable selection program trained end-to-end over a pretrained ensemble to optimize task performance. The proposed framework demonstrates its versatility and effectiveness, outperforming conventional and advanced consensus rules across a variety of classification tasks.
@inproceedings{KVF:ijcai23, author = {Kotary, James and Vito, Vincenzo Di and Fioretto, Ferdinando}, title = {Differentiable Model Selection for Ensemble Learning}, booktitle = {International Joint Conference on Artificial Intelligence}, publisher = {ijcai.org}, pages = {1954--1962}, year = {2023}, url = {https://doi.org/10.24963/ijcai.2023/217}, doi = {10.24963/ijcai.2023/217}, } - IJCAI 2023Backpropagation of Unrolled Solvers with Folded OptimizationJames Kotary, My H. Dinh, and Ferdinando Fioretto
The integration of constrained optimization models as components in deep networks has led to promising advances on many specialized learning tasks. A central challenge in this setting is backpropagation through the solution of an optimization problem, which typically lacks a closed form. One typical strategy is algorithm unrolling, which relies on automatic differentiation through the operations of an iterative solver. While flexible and general, unrolling can encounter accuracy and efficiency issues in practice. These issues can be avoided by analytical differentiation of the optimization, but current frameworks impose rigid requirements on the optimization problem’s form. This paper provides theoretical insights into the backward pass of unrolled optimization, leading to a system for generating efficiently solvable analytical models of backpropagation. Additionally, it proposes a unifying view of unrolling and analytical differentiation through optimization mappings. Experiments over various model-based learning tasks demonstrate the advantages of the approach both computationally and in terms of enhanced expressiveness.
@inproceedings{KDF:ijcai23, author = {Kotary, James and Dinh, My H. and Fioretto, Ferdinando}, title = {Backpropagation of Unrolled Solvers with Folded Optimization}, booktitle = {International Joint Conference on Artificial Intelligence}, publisher = {ijcai.org}, pages = {1963--1970}, year = {2023}, url = {https://doi.org/10.24963/ijcai.2023/218}, doi = {10.24963/ijcai.2023/218}, } - PESGM 2023Load Encoding for Learning AC-OPFTerrence W. K. Mak, Ferdinando Fioretto, and Pascal Van Hentenryck
The AC Optimal Power Flow (AC-OPF) problem is a core building block in electrical transmission system. It seeks the most economical active and reactive generation dispatch to meet demands while satisfying transmission operational limits. It is often solved repeatedly, especially in regions with large penetration of wind farms to avoid violating operational and physical limits. Recent work has shown that deep learning techniques have huge potential in providing accurate approximations of AC-OPF solutions. However, deep learning approaches often suffer from scalability issues, especially when applied to real life power grids. This paper focuses on the scalability limitation and proposes a load compression embedding scheme to reduce training model sizes using a 3-step approach. The approach is evaluated experimentally on large-scale test cases from the PGLib, and produces an order of magnitude improvements in training convergence and prediction accuracy.
@inproceedings{Fioretto:PES23, author = {Mak, Terrence W. K. and Fioretto, Ferdinando and {Van Hentenryck}, Pascal}, title = {Load Encoding for Learning {AC-OPF}}, booktitle = {IEEE Power \& Energy Society General Meeting (PESGM)}, year = {2023}, eprint = {2101.03973}, publisher = {IEEE}, } - AAMAS 2023End-to-End Optimization and Learning for Multiagent EnsemblesJames Kotary, Vincenzo Di Vito, and Ferdinando FiorettoIn International Conference on Autonomous Agents and Multiagent Systems, 2023|Abs|Bib|Online|AlphaXiv
Ensemble learning is an important class of algorithms aiming at creating accurate machine learning models by combining predictions from individual agents. A key challenge for the design of these models is to create effective rules to combine individual predictions for any particular input sample. This paper proposes a unique integration of constrained optimization and learning to derive specialized consensus rules. The paper shows how to derive the ensemble learning task as end-to-end training of a discrete subset selection module. Results over standard benchmarks demonstrate an ability to substantially outperform conventional consensus rules in a variety of settings.
@inproceedings{KVF:aamas23, author = {Kotary, James and Vito, Vincenzo Di and Fioretto, Ferdinando}, title = {End-to-End Optimization and Learning for Multiagent Ensembles}, booktitle = {International Conference on Autonomous Agents and Multiagent Systems}, publisher = {{ACM}}, pages = {2613--2615}, year = {2023}, } - ISGT 2023An Analysis of the Reliability of AC Optimal Power Flow Deep Learning ProxiesMy H. Dinh, Ferdinando Fioretto, Mostafa Mohammadian, and Kyri Baker
Optimal Power Flow (OPF) is a fundamental problem in power systems. It is computationally challenging and a recent line of research has proposed the use of Deep Neural Networks (DNNs) to find OPF approximations at vastly reduced runtimes when compared to those obtained by classical optimization methods. While these works show encouraging results in terms of accuracy and runtime, little is known on why these models can predict OPF solutions accurately, as well as about their robustness. This paper provides a step forward to address this knowledge gap. The paper connects the volatility of the outputs of the generators to the ability of a learning model to approximate them, it sheds light on the characteristics affecting the DNN models to learn good predictors, and it proposes a new model that exploits the observations made by this paper to produce accurate and robust OPF predictions.
@inproceedings{DFMB:isgt23, author = {Dinh, My H. and Fioretto, Ferdinando and Mohammadian, Mostafa and Baker, Kyri}, title = {An Analysis of the Reliability of AC Optimal Power Flow Deep Learning Proxies}, booktitle = {{IEEE} PES Innovative Smart Grid Technologies}, publisher = {}, pages = {}, year = {2023}, url = {https://ieee-isgt-latam.org}, doi = {}, }
2022
- ArXiv 2022Deadwooding: Robust Global Pruning for Deep Neural NetworksSawinder Kaur, Ferdinando Fioretto, and Asif Salekin
The ability of Deep Neural Networks to approximate highly complex functions is key to their success. This benefit, however, comes at the expense of a large model size, which challenges its deployment in resource-constrained environments. Pruning is an effective technique used to limit this issue, but often comes at the cost of reduced accuracy and adversarial robustness. This paper addresses these shortcomings and introduces Deadwooding, a novel global pruning technique that exploits a Lagrangian Dual method to encourage model sparsity while retaining accuracy and ensuring robustness. The resulting model is shown to significantly outperform the state-of-the-art studies in measures of robustness and accuracy.
@article{KFS:22, author = {Kaur, Sawinder and Fioretto, Ferdinando and Salekin, Asif}, title = {Deadwooding: Robust Global Pruning for Deep Neural Networks}, journal = {CoRR}, volume = {abs/2202.05226}, year = {2022}, } - JAIR 2022Proactive Dynamic Distributed Constraint Optimization ProblemsKhoi D. Hoang, Ferdinando Fioretto, Ping Hou, William Yeoh, Makoto Yokoo, and Roie Zivan
The Distributed Constraint Optimization Problem (DCOP) formulation is a powerful tool for modeling multi-agent coordination problems. To solve DCOPs in a dynamic environment, Dynamic DCOPs (D-DCOPs) have been proposed to model the inherent dynamism present in many coordination problems. D-DCOPs solve a sequence of static problems by reacting to changes in the environment as the agents observe them. Such reactive approaches ignore knowledge about future changes of the problem. To overcome this limitation, we introduce Proactive Dynamic DCOPs (PD-DCOPs), a novel formalism to model D-DCOPs in the presence of exogenous uncertainty. In contrast to reactive approaches, PD-DCOPs are able to explicitly model possible changes of the problem and take such information into account when solving the dynamically changing problem in a proactive manner. The additional expressivity of this formalism allows it to model a wider variety of distributed optimization problems. Our work presents both theoretical and practical contributions that advance current dynamic DCOP models: (i) We introduce Proactive Dynamic DCOPs (PD-DCOPs), which explicitly model how the DCOP will change over time; (ii) We develop exact and heuristic algorithms to solve PD-DCOPs in a proactive manner; (iii) We provide theoretical results about the complexity of this new class of DCOPs; and (iv) We empirically evaluate both proactive and reactive algorithms to determine the trade-offs between the two classes. The final contribution is important as our results are the first that identify the characteristics of the problems that the two classes of algorithms excel in.
@article{HFHYY:jairZ22, author = {Hoang, Khoi D. and Fioretto, Ferdinando and Hou, Ping and Yeoh, William and Yokoo, Makoto and Zivan, Roie}, title = {Proactive Dynamic Distributed Constraint Optimization Problems}, journal = {Journal of Artificial Intelligence Research}, volume = {74}, pages = {179--225}, year = {2022}, url = {https://doi.org/10.1613/jair.1.13499}, doi = {10.1613/jair.1.13499}, } - NeurIPS 2022Pruning has a disparate impact on model accuracySpotlightCuong Tran, Ferdinando Fioretto, Jung-Eun Kim, and Rakshit Naidu
Network pruning is a widely-used compression technique that is able to significantly scale down overparameterized models with minimal loss of accuracy. This paper shows that pruning may create or exacerbate disparate impacts. The paper sheds light on the factors to cause such disparities, suggesting differences in gradient norms and distance to decision boundary across groups to be responsible for this critical issue. It analyzes these factors in detail, providing both theoretical and empirical support, and proposes a simple, yet effective, solution that mitigates the disparate impacts caused by pruning.
@inproceedings{TFKN:neurips22, author = {Tran, Cuong and Fioretto, Ferdinando and Kim, Jung-Eun and Naidu, Rakshit}, booktitle = {Advances in Neural Information Processing Systems}, title = {Pruning has a disparate impact on model accuracy}, year = {2022}, volume = {35}, pages = {}, url = {https://openreview.net/forum?id=11nMVZK0WYM}, publisher = {Curran Associates, Inc.}, } - AAAI 2022Fast Approximations for Job Shop Scheduling: A Lagrangian Dual Deep Learning MethodJames Kotary, Ferdinando Fioretto, and Pascal Van Hentenryck
The Jobs Shop Scheduling problem (JSP) is a canonical combinatorial optimization problem that is routinely solved for a variety of industrial purposes. It models the optimal scheduling of multiple sequences of tasks, each under a fixed order of operations, in which individual tasks require exclusive access to a predetermined resource for a specified processing time. The problem is NP-hard and computationally challenging even for medium-sized instances. Motivated by the increased stochasticity in production chains, this paper explores a deep learning approach to deliver efficient and accurate approximations to the JSP. In particular, this paper proposes the design of a deep neural network architecture to exploit the problem structure, its integration with Lagrangian duality to capture the problem constraints, and a post-processing optimization, to guarantee solution feasibility. The resulting method, called JSP-DNN, is evaluated on hard JSP instances from the JSPLIB benchmark library and is shown to produce JSP approximations of high quality at negligible computational costs.
@inproceedings{KFH:aaai22, author = {Kotary, James and Fioretto, Ferdinando and Hentenryck, Pascal Van}, title = {Fast Approximations for Job Shop Scheduling: {A} Lagrangian Dual Deep Learning Method}, booktitle = {AAAI Conference on Artificial Intelligence}, publisher = {{AAAI} Press}, pages = {7239--7246}, year = {2022}, url = {https://ojs.aaai.org/index.php/AAAI/article/view/20685}, } - IJCAI 2022Post-processing of Differentially Private Data: A Fairness PerspectiveKeyu Zhu, Ferdinando Fioretto, and Pascal Van Hentenryck
Post-processing immunity is a fundamental property of differential privacy: it enables arbitrary data-independent transformations to differentially private outputs without affecting their privacy guarantees. Post-processing is routinely applied in data-release applications, including census data, which are then used to make allocations with substantial societal impacts. This paper shows that post-processing causes disparate impacts on individuals or groups and analyzes two critical settings: the release of differentially private datasets and the use of such private datasets for downstream decisions, such as the allocation of funds informed by US Census data. In the first setting, the paper proposes tight bounds on the unfairness for traditional post-processing mechanisms, giving a unique tool to decision makers to quantify the disparate impacts introduced by their release. In the second setting, this paper proposes a novel post-processing mechanism that is (approximately) optimal under different fairness metrics, either reducing fairness issues substantially or reducing the cost of privacy. The theoretical analysis is complemented with numerical simulations on Census data.
@inproceedings{ZFH:ijcai22, author = {Zhu, Keyu and Fioretto, Ferdinando and Hentenryck, Pascal Van}, title = {Post-processing of Differentially Private Data: {A} Fairness Perspective}, booktitle = {International Joint Conference on Artificial Intelligence}, publisher = {ijcai.org}, pages = {4029--4035}, year = {2022}, url = {https://doi.org/10.24963/ijcai.2022/559}, doi = {10.24963/ijcai.2022/559}, } - IJCAI 2022Differential Privacy and Fairness in Decisions and Learning Tasks: A SurveyFerdinando Fioretto, Cuong Tran, Pascal Van Hentenryck, and Keyu Zhu
This paper surveys the recent work in the intersection of differential privacy (DP) and fairness. It focuses on surveying the work observing that DP systems may exacerbate bias and disparate impacts for different groups of individuals. The survey reviews the conditions under which privacy and fairness may be aligned or contrasting goals, analyzes how and why DP exacerbates bias and unfairness in decision problems and learning tasks, and reviews the available solutions to mitigate the fairness issues arising in DP systems. The survey provides a unified understanding of the main challenges and potential risks arising when deploying privacy-preserving machine learning or decisions making tasks under a fairness lens.
@inproceedings{FHZ:ijcai22, author = {Fioretto, Ferdinando and Tran, Cuong and Hentenryck, Pascal Van and Zhu, Keyu}, title = {Differential Privacy and Fairness in Decisions and Learning Tasks: {A} Survey}, booktitle = {International Joint Conference on Artificial Intelligence}, publisher = {ijcai.org}, pages = {5470--5477}, year = {2022}, url = {https://doi.org/10.24963/ijcai.2022/766}, doi = {10.24963/ijcai.2022/766}, } - IJCAI 2022Integrating Machine Learning and Optimization to Boost Decision MakingEarly Career SpotlightFerdinando Fioretto
This paper presents a conceptual review of our recent advancements in the integration of machine learning and optimization. It focuses on describing new hybrid machine learning and optimization methods to predict fast, approximate, solutions to combinatorial problems and to enable structural logical inference.
@inproceedings{F:ijcai22, author = {Fioretto, Ferdinando}, title = {Integrating Machine Learning and Optimization to Boost Decision Making}, booktitle = {International Joint Conference on Artificial Intelligence}, publisher = {ijcai.org}, pages = {5808--5812}, year = {2022}, url = {https://doi.org/10.24963/ijcai.2022/815}, doi = {10.24963/ijcai.2022/815}, } - WWW 2022End-to-End Learning for Fair Ranking SystemsJames Kotary, Ferdinando Fioretto, Pascal Van Hentenryck, and Ziwei Zhu
The learning-to-rank problem aims at ranking items to maximize exposure of those most relevant to a user query. A desirable property of such ranking systems is to guarantee some notion of fairness among specified item groups. While fairness has recently been considered in the context of learning-to-rank systems, current methods cannot provide guarantees on the fairness of the predicted rankings. This paper addresses this gap and introduces Smart Predict and Optimize for Fair Ranking (SPOFR), an integrated optimization and learning framework for fairness-constrained learning to rank. The end-to-end SPOFR framework includes a constrained optimization sub-model and produces ranking policies that are guaranteed to satisfy fairness constraints, while allowing for fine control of the fairness-utility tradeoff. SPOFR is shown to significantly improve on current state-of-the-art fair learning-to-rank systems with respect to established performance metrics.
@inproceedings{KFHZ:WWW22, author = {Kotary, James and Fioretto, Ferdinando and Hentenryck, Pascal Van and Zhu, Ziwei}, title = {End-to-End Learning for Fair Ranking Systems}, booktitle = {The {ACM} Web Conference}, publisher = {{ACM}}, pages = {3520--3530}, year = {2022}, url = {https://doi.org/10.1145/3485447.3512247}, doi = {10.1145/3485447.3512247}, } - PMAPS 2022Differentially-Private Heat and Electricity Markets CoordinationLesia Mitridati, Emma Romei, Gabriela Hug, and Ferdinando Fioretto
Sector coordination between heat and electricity systems has been identified has an energy-efficient and cost-effective way to transition towards a more sustainable energy system. However, the coordination of sequential markets relies on the exchange of sensitive information between the market operators, namely time series of consumers’ loads. To address the privacy concerns arising from this exchange, this paper introduces a novel privacy-preserving Stackelberg mechanism (w-PPSM) which generates differentially-private data streams with high fidelity. The proposed w-PPSM enforces the feasibility and fidelity of the privacy-preserving data with respect to the original problem through a post-processing phase in order to achieve a close-to-optimal coordination between the markets. Multiple numerical simulations in a realistic energy system demonstrate the effectiveness of the w-PPSM, which achieves up to two orders of magnitude reduction in the cost of privacy compared to a traditional differentially-private mechanism.
@inproceedings{MRHF:pmaps22, author = {Mitridati, Lesia and Romei, Emma and Hug, Gabriela and Fioretto, Ferdinando}, title = {Differentially-Private Heat and Electricity Markets Coordination}, booktitle = {International Conference on Probabilistic Methods Applied to Power Systems}, year = {2022}, pages = {1-6}, doi = {10.1109/PMAPS53380.2022.9810564}, } - PMAPS 2022Learning Solutions for Intertemporal Power Systems Optimization with Recurrent Neural NetworksMostafa Mohammadian, Kyri Baker, My H. Dinh, and Ferdinando Fioretto
Learning mappings between system loading and optimal dispatch solutions has been a recent topic of interest in the power systems and machine learning communities. However, previous works have ignored practical power system constraints such as generator ramp limits and other intertemporal requirements. Additionally, optimal power flow runs are not performed independently of previous timesteps - in most cases, an OPF solution representing the current state of the system is heavily related to the OPF solution from previous timesteps. In this paper, we train a recurrent neural network, which embeds natural relationships between timesteps, to predict the optimal solution of convex power systems optimization problems with intertemporal constraints. In contrast to traditional forecasting methods, the computational benefits from this technique can allow operators to rapidly simulate forecasts of system operation and corresponding optimal solutions to provide a more comprehensive view of future system states.
@inproceedings{MBDF:pmaps22, author = {Mohammadian, Mostafa and Baker, Kyri and Dinh, My H. and Fioretto, Ferdinando}, title = {Learning Solutions for Intertemporal Power Systems Optimization with Recurrent Neural Networks}, booktitle = {International Conference on Probabilistic Methods Applied to Power Systems}, year = {2022}, pages = {1-6}, doi = {10.1109/PMAPS53380.2022.9810638}, }
2021
- AIJ 2021Differential Privacy of Hierarchical Census Data: An Optimization ApproachInvited in the IJCAI-21 Journal TrackFerdinando Fioretto, Pascal Van Hentenryck, and Keyu Zhu
This paper is motivated by applications of a Census Bureau interested in releasing aggregate socio-economic data about a large population without revealing sensitive information about any individual. The released information can be the number of individuals living alone, the number of cars they own, or their salary brackets. Recent events have identified some of the privacy challenges faced by these organizations [1]. To address them, this paper presents a novel differential-privacy mechanism for releasing hierarchical counts of individuals. The counts are reported at multiple granularities (e.g., the national, state, and county levels) and must be consistent across all levels. The core of the mechanism is an optimization model that redistributes the noise introduced to achieve differential privacy in order to meet the consistency constraints between the hierarchical levels. The key technical contribution of the paper shows that this optimization problem can be solved in polynomial time by exploiting the structure of its cost functions. Experimental results on very large, real datasets show that the proposed mechanism provides improvements of up to two orders of magnitude in terms of computational efficiency and accuracy with respect to other state-of-the-art techniques.
@article{FvHZ:AIJ21, title = {Differential Privacy of Hierarchical Census Data: An Optimization Approach}, author = {Fioretto, Ferdinando and {Van Hentenryck}, Pascal and Zhu, Keyu}, journal = {Artificial Intelligence Journal}, volume = {296}, pages = {103475}, year = {2021}, url = {https://doi.org/10.1016/j.artint.2021.103475}, doi = {10.1016/j.artint.2021.103475}, } - TPWRS 2021Differentially Private Optimal Power Flow for Distribution Grids2022 TPWRS Best Paper AwardVladimir Dvorkin, Ferdinando Fioretto, Pascal Van Hentenryck, Pierre Pinson, and Jalal Kazempour
Although distribution grid customers are obliged to share their consumption data with distribution system operators (DSOs), a possible leakage of this data is often disregarded in operational routines of DSOs. This paper introduces a privacy-preserving optimal power flow (OPF) mechanism for distribution grids that secures customer privacy from unauthorised access to OPF solutions, e.g., current and voltage measurements. The mechanism is based on the framework of differential privacy that allows to control the participation risks of individuals in a dataset by applying a carefully calibrated noise to the output of a computation. Unlike existing private mechanisms, this mechanism does not apply the noise to the optimization parameters or its result. Instead, it optimizes OPF variables as affine functions of the random noise, which weakens the correlation between the grid loads and OPF variables. To ensure feasibility of the randomized OPF solution, the mechanism makes use of chance constraints enforced on the grid limits. The mechanism is further extended to control the optimality loss induced by the random noise, as well as the variance of OPF variables. The paper shows that the differentially private OPF solution does not leak customer loads up to specified parameters.
@article{DFvHPK:tpwrs21, author = {Dvorkin, Vladimir and Fioretto, Ferdinando and {Van Hentenryck}, Pascal and Pinson, Pierre and Kazempour, Jalal}, title = {Differentially Private Optimal Power Flow for Distribution Grids}, journal = {IEEE Transactions on Power Systems}, year = {2021}, volume = {36}, number = {3}, pages = {2186--2196}, doi = {10.1109/TPWRS.2020.3031314}, url = {https://ieeexplore.ieee.org/document/9226144}, } - NeurIPS 2021Differentially Private Empirical Risk Minimization under the Fairness LensCuong Tran, My H. Dinh, and Ferdinando Fioretto
Differential Privacy (DP) is an important privacy-enhancing technology for private machine learning systems. It allows to measure and bound the risk associated with an individual participation in a computation. However, it was recently observed that DP learning systems may exacerbate bias and unfairness for different groups of individuals. This paper builds on these important observations and sheds light on the causes of the disparate impacts arising in the problem of differentially private empirical risk minimization. It focuses on the accuracy disparity arising among groups of individuals in two well-studied DP learning methods: output perturbation and differentially private stochastic gradient descent. The paper analyzes which data and model properties are responsible for the disproportionate impacts, why these aspects are affecting different groups disproportionately and proposes guidelines to mitigate these effects. The proposed approach is evaluated on several datasets and settings.
@inproceedings{TMF:neurips21, author = {Tran, Cuong and H.~Dinh, My and Fioretto, Ferdinando}, title = {Differentially Private Empirical Risk Minimization under the Fairness Lens}, booktitle = {Advances in Neural Information Processing Systems}, publisher = {Curran Associates, Inc.}, year = {2021}, volume = {34}, pages = {27555--27565}, url = {https://openreview.net/forum?id=7EFdodSWee4}, } - NeurIPS 2021Learning Hard Optimization Problems: A Data Generation PerspectiveJames Kotary, Ferdinando Fioretto, and Pascal Van Hentenryck
Optimization problems are ubiquitous in our societies and are present in almost every segment of the economy. Most of these optimization problems are NP-hard and computationally demanding, often requiring approximate solutions for large-scale instances. Machine learning frameworks that learn to approximate solutions to such hard optimization problems are a potentially promising avenue to address these difficulties, particularly when many closely related problem instances must be solved repeatedly. Supervised learning frameworks can train a model using the outputs of pre-solved instances. However, when the outputs are themselves approximations, when the optimization problem has symmetric solutions, and/or when the solver uses randomization, solutions to closely related instances may exhibit large differences and the learning task can become inherently more difficult. This paper demonstrates this critical challenge, connects the volatility of the training data to the ability of a model to approximate it, and proposes a method for producing (exact or approximate) solutions to optimization problems that are more amenable to supervised learning tasks. The effectiveness of the method is tested on hard non-linear nonconvex and discrete combinatorial problems.
@inproceedings{KFvH:neurips21, author = {Kotary, James and Fioretto, Ferdinando and {Van Hentenryck}, Pascal}, title = {Learning Hard Optimization Problems: A Data Generation Perspective}, booktitle = {Advances in Neural Information Processing Systems}, publisher = {Curran Associates, Inc.}, year = {2021}, volume = {34}, pages = {24981--24992}, url = {https://openreview.net/forum?id=2zO2lb7ykMD}, } - AAAI 2021Differentially Private and Fair Deep Learning: A Lagrangian Dual ApproachCuong Tran, Ferdinando Fioretto, and Pascal Van HentenryckAlso appeared in: PPAI (AAAI 2021)
A critical concern in data-driven decision making is to build models whose outcomes do not discriminate against some demographic groups, including gender, ethnicity, or age. To ensure non-discrimination in learning tasks, knowledge of the sensitive attributes is essential, while, in practice, these attributes may not be available due to legal and ethical requirements. To address this challenge, this paper studies a model that protects the privacy of the individuals’ sensitive information while also allowing it to learn non-discriminatory predictors. The method relies on the notion of differential privacy and the use of Lagrangian duality to design neural networks that can accommodate fairness constraints while guaranteeing the privacy of sensitive attributes. The paper analyses the tension between accuracy, privacy, and fairness and the experimental evaluation illustrates the benefits of the proposed model on several prediction tasks.
@inproceedings{TFvH:aaai21, author = {Tran, Cuong and Fioretto, Ferdinando and {Van Hentenryck}, Pascal}, title = {Differentially Private and Fair Deep Learning: {A} Lagrangian Dual Approach}, booktitle = {AAAI Conference on Artificial Intelligence}, publisher = {{AAAI} Press}, pages = {9932--9939}, year = {2021}, url = {https://ojs.aaai.org/index.php/AAAI/article/view/17193}, } - AAAI 2021Bias and Variance of Post-processing in Differential PrivacyKeyu Zhu, Pascal Van Hentenryck, and Ferdinando Fioretto
Post-processing immunity is a fundamental property of differential privacy: it enables the application of arbitrary data-independent transformations to the results of differentially private outputs without affecting their privacy guarantees. When query outputs must satisfy domain constraints, post-processing can be used to project them back onto the feasibility region. Moreover, when the feasible region is convex, a widely adopted class of post-processing steps is also guaranteed to improve accuracy. Post-processing has been applied successfully in many applications including census data, energy systems, and mobility. However, its effects on the noise distribution is poorly understood: It is often argued that post-processing may introduce bias and increase variance. This paper takes a first step towards understanding the properties of post-processing. It considers the release of census data and examines, both empirically and theoretically, the behavior of a widely adopted class of post-processing functions.
@inproceedings{ZHF:aaai21, author = {Zhu, Keyu and Hentenryck, Pascal Van and Fioretto, Ferdinando}, title = {Bias and Variance of Post-processing in Differential Privacy}, booktitle = {AAAI Conference on Artificial Intelligence}, pages = {11177--11184}, publisher = {{AAAI} Press}, year = {2021}, url = {https://ojs.aaai.org/index.php/AAAI/article/view/17333}, } - IJCAI 2021Decision Making with Differential Privacy under a Fairness Lens2022 Caspar Bowden PET awardCuong Tran, Ferdinando Fioretto, Pascal Van Hentenryck, and Zhiyan YaoTPDP (NeurIPS 2021)
Agencies, such as the U.S. Census Bureau, release data sets and statistics about groups of individuals that are used as input to a number of critical decision processes. To conform to privacy and confidentiality requirements, these agencies are often required to release privacy-preserving versions of the data. This paper studies the release of differentially private data sets and analyzes their impact on some critical resource allocation tasks under a fairness perspective. The paper shows that, when the decisions take as input differentially private data, the noise added to achieve privacy disproportionately impacts some groups over others. The paper analyzes the reasons for these disproportionate impacts and proposes guidelines to mitigate these effects. The proposed approaches are evaluated on critical decision problems that use differentially private census data.
@inproceedings{TFHY:ijcai21, author = {Tran, Cuong and Fioretto, Ferdinando and {Van Hentenryck}, Pascal and Yao, Zhiyan}, title = {Decision Making with Differential Privacy under a Fairness Lens}, booktitle = {International Joint Conference on Artificial Intelligence}, publisher = {ijcai.org}, pages = {560--566}, year = {2021}, url = {https://doi.org/10.24963/ijcai.2021/78}, doi = {10.24963/ijcai.2021/78}, } - IJCAI 2021End-to-End Constrained Optimization Learning: A SurveyJames Kotary, Ferdinando Fioretto, Pascal Van Hentenryck, and Bryan Wilder
This paper surveys the recent attempts at leveraging machine learning to solve constrained optimization problems. It focuses on surveying the work on integrating combinatorial solvers and optimization methods with machine learning architectures. These approaches hold the promise to develop new hybrid machine learning and optimization methods to predict fast, approximate, solutions to combinatorial problems and to enable structural logical inference. This paper presents a conceptual review of the recent advancements in this emerging area.
@inproceedings{KFHW:ijcai21, author = {Kotary, James and Fioretto, Ferdinando and {Van Hentenryck}, Pascal and Wilder, Bryan}, title = {End-to-End Constrained Optimization Learning: {A} Survey}, booktitle = {International Joint Conference on Artificial Intelligence}, publisher = {ijcai.org}, pages = {4475--4482}, year = {2021}, url = {https://doi.org/10.24963/ijcai.2021/610}, doi = {10.24963/ijcai.2021/610}, } - AAMAS 2021Privacy-Preserving and Accountable Multi-agent LearningAnudit Nagar, Cuong Tran, and Ferdinando FiorettoIn International Conference on Autonomous Agents and Multiagent Systems, 2021|Abs|Bib|Online|AlphaXiv
Distributed multi-agent learning enables agents to cooperatively train a model without requiring to share their datasets. While this setting ensures some level of privacy, it has been shown that, even when data is not directly shared, the training process is vulnerable to privacy attacks including data reconstruction and model inversion attacks. Additionally, malicious agents that train on inverted labels or random data, may arbitrarily weaken the accuracy of the global model. This paper addresses these challenges and presents Privacy-preserving and Accountable Distributed Learning (PA-DL), a fully decentralized framework that relies on Differential Privacy to guarantee strong privacy protection of the agents data, and Ethereum smart contracts to ensure accountability.
@inproceedings{NTF:aamas21, author = {Nagar, Anudit and Tran, Cuong and Fioretto, Ferdinando}, title = {Privacy-Preserving and Accountable Multi-agent Learning}, booktitle = {International Conference on Autonomous Agents and Multiagent Systems}, publisher = {{ACM}}, pages = {1605--1606}, year = {2021}, } - CP 2021Ferdinando Fioretto
@inproceedings{F:cp21, author = {Fioretto, Ferdinando}, title = {Constrained-Based Differential Privacy (Invited Talk)}, booktitle = {International Conference on Principles and Practice of Constraint Programming}, series = {LIPIcs}, volume = {210}, pages = {2:1--2:1}, publisher = {Schloss Dagstuhl}, year = {2021}, url = {https://doi.org/10.4230/LIPIcs.CP.2021.2}, doi = {10.4230/LIPIcs.CP.2021.2}, }
2020
- preprint 2020Terrence W. K. Mak, Ferdinando Fioretto, and Pascal Van Hentenryck
@article{Fioretto:BilevelDP20, title = {Bilevel Optimization for Differentially Private Optimization in Energy Systems}, year = {2020}, doi = {10.48550/arXiv.2001.09508}, author = {Mak, Terrence W. K. and Fioretto, Ferdinando and Van Hentenryck, Pascal}, journal = {CoRR}, volume = {abs/2001.09508}, } - preprint 2020Differentially Private Convex Optimization with Feasibility GuaranteesVladimir Dvorkin, Ferdinando Fioretto, Pascal Van Hentenryck, Jalal Kazempour, and Pierre Pinson
This paper develops a novel differentially private framework to solve convex optimization problems with sensitive optimization data and complex physical or operational constraints. Unlike standard noise-additive algorithms, that act primarily on the problem data, objective or solution, and disregard the problem constraints, this framework requires the optimization variables to be a function of the noise and exploits a chance-constrained problem reformulation with formal feasibility guarantees. The noise is calibrated to provide differential privacy for identity and linear queries on the optimization solution. For many applications, including resource allocation problems, the proposed framework provides a trade-off between the expected optimality loss and the variance of optimization results.
@article{DFvHKP:20b, author = {Dvorkin, Vladimir and Fioretto, Ferdinando and Hentenryck, Pascal Van and Kazempour, Jalal and Pinson, Pierre}, title = {Differentially Private Convex Optimization with Feasibility Guarantees}, journal = {CoRR}, volume = {abs/2006.12338}, year = {2020}, } - ArXiv 2020High-Fidelity Machine Learning Approximations of Large-Scale Optimal Power FlowMinas Chatzos, Ferdinando Fioretto, Terrence W. K. Mak, and Pascal Van Hentenryck
The AC Optimal Power Flow (AC-OPF) is a key building block in many power system applications. It determines generator setpoints at minimal cost that meet the power demands while satisfying the underlying physical and operational constraints. It is non-convex and NP-hard, and computationally challenging for large-scale power systems. Motivated by the increased stochasticity in generation schedules and increasing penetration of renewable sources, this paper explores a deep learning approach to deliver highly efficient and accurate approximations to the AC-OPF. In particular, the paper proposes an integration of deep neural networks and Lagrangian duality to capture the physical and operational constraints. The resulting model, called OPF-DNN, is evaluated on real case studies from the French transmission system, with up to 3,400 buses and 4,500 lines. Computational results show that OPF-DNN produces highly accurate AC-OPF approximations whose costs are within 0.01% of optimality. OPF-DNN generates, in milliseconds, solutions that capture the problem constraints with high fidelity.
@article{CFMvH:20, author = {Chatzos, Minas and Fioretto, Ferdinando and Mak, Terrence W. K. and Hentenryck, Pascal Van}, title = {High-Fidelity Machine Learning Approximations of Large-Scale Optimal Power Flow}, journal = {CoRR}, volume = {abs/2006.16356}, year = {2020}, } - AI Mag. 2020Grace Bang, Guy Barash, Ryan Beal, Jacques Calı̀, Mauricio Castillo-Effen, Xin Cynthia Chen, Niyati Chhaya, Rachel Cummings, Rohan Dhoopar, Sebastijan Dumancic, Huáscar Espinoza, Eitan Farchi, Ferdinando Fioretto, Raquel Fuentetaja, Christopher William Geib, Odd Erik Gundersen, José Hernández-Orallo, Xiaowei Huang, Kokil Jaidka, Sarah Keren, Seokhwan Kim, Michel Galley, Xiaomo Liu, Tyler Lu, Zhiqiang Ma, Richard Mallah, John A. McDermid, Martin Michalowski, Reuth Mirsky, Seán Ó hÉigeartaigh, Deepak Ramachandran, Javier Segovia Aguas, Onn Shehory, Arash Shaban-Nejad, Vered Shwartz, Siddharth Srivastava, Kartik Talamadupula, Jian Tang, Pascal Van Hentenryck, Dell Zhang, and Jian Zhang
@article{DBLP:journals/aim/BangBBCCCCCDDEF20, author = {Bang, Grace and Barash, Guy and Beal, Ryan and Cal{\`{\i}}, Jacques and Castillo{-}Effen, Mauricio and Chen, Xin Cynthia and Chhaya, Niyati and Cummings, Rachel and Dhoopar, Rohan and Dumancic, Sebastijan and Espinoza, Hu{\'{a}}scar and Farchi, Eitan and Fioretto, Ferdinando and Fuentetaja, Raquel and Geib, Christopher William and Gundersen, Odd Erik and Hern{\'{a}}ndez{-}Orallo, Jos{\'{e}} and Huang, Xiaowei and Jaidka, Kokil and Keren, Sarah and Kim, Seokhwan and Galley, Michel and Liu, Xiaomo and Lu, Tyler and Ma, Zhiqiang and Mallah, Richard and McDermid, John A. and Michalowski, Martin and Mirsky, Reuth and h{\'{E}}igeartaigh, Se{\'{a}}n {\'{O}} and Ramachandran, Deepak and Aguas, Javier Segovia and Shehory, Onn and Shaban{-}Nejad, Arash and Shwartz, Vered and Srivastava, Siddharth and Talamadupula, Kartik and Tang, Jian and Hentenryck, Pascal Van and Zhang, Dell and Zhang, Jian}, title = {The Association for the Advancement of Artificial Intelligence 2020 Workshop Program}, journal = {{AI} Magazine}, volume = {41}, number = {4}, pages = {100--114}, year = {2020}, url = {https://doi.org/10.1609/aimag.v41i4.7398}, doi = {10.1609/aimag.v41i4.7398}, } - TSG 2020Differential Privacy for Power Grid ObfuscationFerdinando Fioretto, Terrence W. K. Mak, and Pascal Van Hentenryck
The availability of high-fidelity energy networks brings significant value to academic and commercial research. However, such releases also raise fundamental concerns related to privacy and security as they can reveal sensitive commercial information and expose system vulnerabilities. This paper investigates how to release the data for power networks where the parameters of transmission lines and transformers are obfuscated. It does so by using the framework of Differential Privacy (DP), that provides strong privacy guarantees and has attracted significant attention in recent years. Unfortunately, simple DP mechanisms often result in AC-infeasible networks. To address these concerns, this paper presents a novel differentially private mechanism that guarantees AC-feasibility and largely preserves the fidelity of the obfuscated power network. Experimental results also show that the obfuscation significantly reduces the potential damage of an attack carried by exploiting the released dataset.
@article{FMH:tsg20, author = {Fioretto, Ferdinando and Mak, Terrence W. K. and {Van Hentenryck}, Pascal}, title = {Differential Privacy for Power Grid Obfuscation}, journal = {IEEE Transactions on Smart Grids}, volume = {11}, number = {2}, pages = {1356--1366}, year = {2020}, url = {https://doi.org/10.1109/TSG.2019.2936712}, doi = {10.1109/TSG.2019.2936712}, } - TPWRS 2020Privacy-Preserving Power System Obfuscation: A Bilevel Optimization Approach2021 TPWRS Best Paper AwardTerrence W. K. Mak, Ferdinando Fioretto, Lyndon Shi, and Pascal Van Hentenryck
This paper considers the problem of releasing optimal power flow (OPF) test cases that preserve the privacy of customers (loads) using the notion of Differential Privacy. It is motivated by the observation that traditional differential privacy algorithms are not suitable for releasing privacy preserving OPF test cases: The added noise fundamentally changes the nature of the underlying optimization and often leads to test cases with no solutions. To remedy this limitation, the paper introduces the OPF Load Indistinguishability (OLI) problem, which guarantees load privacy while satisfying the OPF constraints and remaining close to the optimal dispatch cost. The paper introduces an exact mechanism, based on bilevel optimization, as well as three mechanisms that approximate the OLI problem accurately. These mechanisms enjoy desirable theoretical properties, and the computational experiments show that they produce orders of magnitude improvements over standard approaches on an extensive collection of test cases.
- AAAI 2020Predicting AC Optimal Power Flows: Combining Deep Learning and Lagrangian Dual MethodsFerdinando Fioretto, Terrence W. K. Mak, and Pascal Van Hentenryck
The Optimal Power Flow (OPF) problem is a fundamental building block for the optimization of electrical power systems. It is nonlinear and nonconvex and computes the generator setpoints for power and voltage, given a set of load demands. It is often solved repeatedly under various conditions, either in real-time or in large-scale studies. This need is further exacerbated by the increasing stochasticity of power systems due to renewable energy sources in front and behind the meter. To address these challenges, this paper presents a deep learning approach to the OPF. The learning model exploits the information available in the similar states of the system (which is commonly available in practical applications), as well as a dual Lagrangian method to satisfy the physical and engineering constraints present in the OPF. The proposed model is evaluated on a large collection of realistic medium-sized power systems. The experimental results show that its predictions are highly accurate with average errors as low as 0.2%. Additionally, the proposed approach is shown to improve the accuracy of the widely adopted linear DC approximation by at least two orders of magnitude.
@inproceedings{FMH:aaai20, author = {Fioretto, Ferdinando and Mak, Terrence W. K. and Hentenryck, Pascal Van}, title = {Predicting {AC} Optimal Power Flows: Combining Deep Learning and Lagrangian Dual Methods}, booktitle = {AAAI Conference on Artificial Intelligence}, pages = {630--637}, publisher = {{AAAI} Press}, year = {2020}, url = {https://ojs.aaai.org/index.php/AAAI/article/view/5403}, } - IJCAI 2020Differential Privacy for Stackelberg GamesFerdinando Fioretto, Lesia Mitridati, and Pascal Van Hentenryck
This paper introduces a differentially private (DP) mechanism to protect the information exchanged during the coordination of sequential and interdependent markets. This coordination represents a classic Stackelberg game and relies on the exchange of sensitive information between the system agents. The paper is motivated by the observation that the perturbation introduced by traditional DP mechanisms fundamentally changes the underlying optimization problem and even leads to unsatisfiable instances. To remedy such limitation, the paper introduces the Privacy-Preserving Stackelberg Mechanism (PPSM), a framework that enforces the notions of feasibility and fidelity of the privacy-preserving information to the original problem objective. PPSM complies with the notion of differential privacy and ensures that the outcomes of the privacy-preserving coordination mechanism are close-to-optimality for each agent. Experimental results on several gas and electricity market benchmarks based on a real case study demonstrate the effectiveness of the approach.
@inproceedings{FMH:ijcai20, author = {Fioretto, Ferdinando and Mitridati, Lesia and {Van Hentenryck}, Pascal}, title = {Differential Privacy for Stackelberg Games}, booktitle = {International Joint Conference on Artificial Intelligence}, pages = {3480--3486}, publisher = {ijcai.org}, year = {2020}, url = {https://doi.org/10.24963/ijcai.2020/481}, doi = {10.24963/ijcai.2020/481}, } - IJCAI 2020OptStream: Releasing Time Series Privately (Extended Abstract)Ferdinando Fioretto, and Pascal Van HentenryckIn International Joint Conference on Artificial Intelligence, 2020Appeared in JAIR 2019|Abs|Bib|Online|AlphaXiv
Many applications of machine learning and optimization operate on data streams. While these datasets are fundamental to fuel decision-making algorithms, often they contain sensitive information about individuals, and their usage poses significant privacy risks. Motivated by an application in energy systems, this paper presents OptStream, a novel algorithm for releasing differentially private data streams under the w-event model of privacy. OptStream is a 4-step procedure consisting of sampling, perturbation, reconstruction, and post-processing modules. First, the sampling module selects a small set of points to access in each period of interest. Then, the perturbation module adds noise to the sampled data points to guarantee privacy. Next, the reconstruction module reassembles non-sampled data points from the perturbed sample points. Finally, the post-processing module uses convex optimization over the privacy-preserving output of the previous modules, as well as the privacy-preserving answers of additional queries on the data stream, to improve accuracy by redistributing the added noise. OptStream is evaluated on a test case involving the release of a real data stream from the largest European transmission operator. Experimental results show that OptStream may not only improve the accuracy of state-of-the-art methods by at least one order of magnitude but also supports accurate load forecasting on the privacy-preserving data.
@inproceedings{FvH:IJCAI20, author = {Fioretto, Ferdinando and Hentenryck, Pascal Van}, title = {OptStream: Releasing Time Series Privately (Extended Abstract)}, booktitle = {International Joint Conference on Artificial Intelligence}, pages = {5135--5139}, publisher = {ijcai.org}, year = {2020}, url = {https://doi.org/10.24963/ijcai.2020/722}, doi = {10.24963/ijcai.2020/722}, note = {Appeared in JAIR 2019} } - ECML 2020Lagrangian Duality for Constrained Deep LearningFerdinando Fioretto, Pascal Van Hentenryck, Terrence W. K. Mak, Cuong Tran, Federico Baldo, and Michele Lombardi
This paper explores the potential of Lagrangian duality for learning applications that feature complex constraints. Such constraints arise in many science and engineering domains, where the task amounts to learning optimization problems which must be solved repeatedly and include hard physical and operational constraints. The paper also considers applications where the learning task must enforce constraints on the predictor itself, either because they are natural properties of the function to learn or because it is desirable from a societal standpoint to impose them. This paper demonstrates experimentally that Lagrangian duality brings significant benefits for these applications. In energy domains, the combination of Lagrangian duality and deep learning can be used to obtain state-of-the-art results to predict optimal power flows, in energy systems, and optimal compressor settings, in gas networks. In transprecision computing, Lagrangian duality can complement deep learning to impose monotonicity constraints on the predictor without sacrificing accuracy. Finally, Lagrangian duality can be used to enforce fairness constraints on a predictor and obtain state-of-the-art results when minimizing disparate treatments.
@inproceedings{FvHMTBL:ecml20, author = {Fioretto, Ferdinando and {Van Hentenryck}, Pascal and Mak, Terrence W. K. and Tran, Cuong and Baldo, Federico and Lombardi, Michele}, title = {Lagrangian Duality for Constrained Deep Learning}, booktitle = {European Conference on Machine Learning}, series = {Lecture Notes in Computer Science}, volume = {12461}, pages = {118--135}, publisher = {Springer}, year = {2020}, url = {https://doi.org/10.1007/978-3-030-67670-4_8}, doi = {10.1007/978-3-030-67670-4_8}, } - PSCC 2020Privacy-preserving obfuscation for distributed power systemsTerrence W.K. Mak, Ferdinando Fioretto, and Pascal Van Hentenryck
This paper considers the problem of releasing privacy-preserving load data of a decentralized operated power system. The paper focuses on data used to solve Optimal Power Flow (OPF) problems and proposes a distributed algorithm that complies with the notion of Differential Privacy, a strong privacy framework used to bound the risk of re-identification. The problem is challenging since the application of traditional differential privacy mechanisms to the load data fundamentally changes the nature of the underlying optimization problem and often leads to severe feasibility issues. The proposed differentially private distributed algorithm is based on the Alternating Direction Method of Multipliers (ADMM) and guarantees that the released privacy-preserving data retains high fidelity and satisfies the AC power flow constraints. Experimental results on a variety of OPF benchmarks demonstrate the effectiveness of the approach.
@article{MFvH:pscc20, author = {Mak, Terrence W.K. and Fioretto, Ferdinando and {Van Hentenryck}, Pascal}, title = {Privacy-preserving obfuscation for distributed power systems}, journal = {Electric Power Systems Research}, volume = {189}, pages = {106718}, year = {2020}, issn = {0378-7796}, doi = {10.1016/j.epsr.2020.106718}, url = {https://www.sciencedirect.com/science/article/pii/S0378779620305216} } - PRIMA 2020The Smart Appliance Scheduling Problem: A Bayesian Optimization ApproachAtena M. Tabakhi, William Yeoh, and Ferdinando Fioretto
Daily energy demand peaks induce high greenhouse gas emissions and are deleterious to the power grid operations. The autonomous and coordinated control of smart appliances in residential buildings represents an effective solution to reduce peak demands. This coordination problem is challenging as it involves, not only, scheduling devices to minimize energy peaks, but also to comply with user’ preferences. Prior work assumed these preferences to be fully specified and known a priori, which is, however, unrealistic. To remedy this limitation, this paper introduces a Bayesian optimization approach for smart appliance scheduling when the users’ satisfaction with a schedule must be elicited, and thus considered expensive to evaluate. The paper presents a set of ad-hoc energy-cost based acquisition functions to drive the Bayesian optimization problem to find schedules that maximize the user’s satisfaction. The experimental results demonstrate the effectiveness of the proposed energy-cost based acquisition functions which improve the algorithm’s performance up to 26%.
@inproceedings{TYF:prima20, author = {Tabakhi, Atena M. and Yeoh, William and Fioretto, Ferdinando}, title = {The Smart Appliance Scheduling Problem: {A} Bayesian Optimization Approach}, booktitle = {Principles and Practice of Multi-Agent Systems}, series = {Lecture Notes in Computer Science}, volume = {12568}, pages = {100--115}, publisher = {Springer}, year = {2020}, url = {https://link.springer.com/chapter/10.1007/978-3-030-69322-0_7}, doi = {10.1007/978-3-030-69322-0_7}, }
2019
- JAIR 2019OptStream: Releasing Time Series PrivatelyFerdinando Fioretto, and Pascal Van Hentenryck
Many applications of machine learning and optimization operate on data streams. While these datasets are fundamental to fuel decision-making algorithms, often they contain sensitive information about individuals, and their usage poses significant privacy risks. Motivated by an application in energy systems, this paper presents OptStream, a novel algorithm for releasing differentially private data streams under the w-event model of privacy. OptStream is a 4-step procedure consisting of sampling, perturbation, reconstruction, and post-processing modules. First, the sampling module selects a small set of points to access in each period of interest. Then, the perturbation module adds noise to the sampled data points to guarantee privacy. Next, the reconstruction module re-assembles non-sampled data points from the perturbed sample points. Finally, the post-processing module uses convex optimization over the privacy-preserving output of the previous modules, as well as the privacy-preserving answers of additional queries on the data stream, to improve accuracy by redistributing the added noise. OptStream is evaluated on a test case involving the release of a real data stream from the largest European transmission operator. Experimental results show that OptStream may not only improve the accuracy of state-of-the-art methods by at least one order of magnitude but also supports accurate load forecasting on the privacy-preserving data.
@article{FvH:jair19, author = {Fioretto, Ferdinando and Hentenryck, Pascal Van}, title = {OptStream: Releasing Time Series Privately}, journal = {Journal of Artificial Intelligence Research}, volume = {65}, pages = {423--456}, year = {2019}, url = {https://doi.org/10.1613/jair.1.11583}, doi = {10.1613/jair.1.11583}, } - AAMAS 2019Privacy-Preserving Federated Data SharingFerdinando Fioretto, and Pascal Van Hentenryck
Consider a set of agents with sensitive datasets who are interested in the same prediction task and would like to share their datasets without revealing private information. For instance, the agents may be medical centers with their own historical databases and the task may be the diagnosis of a rare form of a disease. This paper investigates whether sharing privacy-preserving versions of these datasets may improve the agent predictions. It proposes a Privacy-preserving Federated Data Sharing (PFDS) protocol that each agent can run locally to produce a privacy-preserving version of its original dataset. The PFDS protocol is evaluated on several standard prediction tasks and experimental results demonstrate the potential of sharing privacy- preserving datasets to produce accurate predictors.
@inproceedings{FvH:aamas19, author = {Fioretto, Ferdinando and Hentenryck, Pascal Van}, title = {Privacy-Preserving Federated Data Sharing}, booktitle = {International Conference on Autonomous Agents and Multiagent Systems}, pages = {638--646}, publisher = {International Foundation for Autonomous Agents and Multiagent Systems}, year = {2019}, } - CP 2019Differential Privacy of Hierarchical Census Data: An Optimization ApproachInvited to Constraint journal - selected paperFerdinando Fioretto, and Pascal Van HentenryckIn International Conference on Principles and Practice of Constraint Programming, 2019|Abs|Bib|Online
This paper is motivated by applications of a Census Bureau interested in releasing aggregate socio-economic data about a large population without revealing sensitive information. The released information can be the number of individuals living alone, the number of cars they own, or their salary brackets. Recent events have identified some of the privacy challenges faced by these organizations. To address them, this paper presents a novel differential-privacy mechanism for releasing hierarchical counts of individuals satisfying a given property. The counts are reported at multiple granularities (e.g., the national, state, and county levels) and must be consistent across levels. The core of the mechanism is an optimization model that redistributes the noise introduced to attain privacy in order to meet the consistency constraints between the hierarchical levels. The key technical contribution of the paper shows that this optimization problem can be solved in polynomial time by exploiting the structure of its cost functions. Experimental results on very large, real datasets show that the proposed mechanism provides improvements up to two orders of magnitude in terms of computational efficiency and accuracy with respect to other state-of-the-art techniques.
@inproceedings{FvH:cp19, author = {Fioretto, Ferdinando and Hentenryck, Pascal Van}, title = {Differential Privacy of Hierarchical Census Data: An Optimization Approach}, booktitle = {International Conference on Principles and Practice of Constraint Programming}, series = {Lecture Notes in Computer Science}, volume = {11802}, pages = {639--655}, publisher = {Springer}, year = {2019}, url = {https://doi.org/10.1007/978-3-030-30048-7_37}, doi = {10.1007/978-3-030-30048-7_37}, } - IJCAI 2019Privacy-Preserving Obfuscation of Critical Infrastructure NetworksFerdinando Fioretto, Terrence W. K. Mak, and Pascal Van Hentenryck
The paper studies how to release data about a critical infrastructure network (e.g., a power network or a transportation network) without disclosing sensitive information that can be exploited by malevolent agents, while preserving the realism of the network. It proposes a novel obfuscation mechanism that combines several privacy-preserving building blocks with a bi-level optimization model to significantly improve accuracy. The obfuscation is evaluated for both realism and privacy properties on real energy and transportation networks. Experimental results show the obfuscation mechanism substantially reduces the potential damage of an attack exploiting the released data to harm the real network.
@inproceedings{FMvH:ijcai19, author = {Fioretto, Ferdinando and Mak, Terrence W. K. and Hentenryck, Pascal Van}, editor = {Kraus, Sarit}, title = {Privacy-Preserving Obfuscation of Critical Infrastructure Networks}, booktitle = {International Joint Conference on Artificial Intelligence}, pages = {1086--1092}, publisher = {ijcai.org}, year = {2019}, url = {https://doi.org/10.24963/ijcai.2019/152}, doi = {10.24963/ijcai.2019/152}, }
2018
- Documentation 2018Russell Bent, Carleton Coffrin, Ferdinando Fioretto, Terrence W. K. Mak, Patrick Panciatici, and Pascal Van Hentenryck
@techreport{Fioretto:GRG18, title = {GRG-Schema v4.0}, year = {2018}, author = {Bent, Russell and Coffrin, Carleton and Fioretto, Ferdinando and Mak, Terrence W. K. and Panciatici, Patrick and Van Hentenryck, Pascal}, institution = {Grid Research for Good (GRG)}, note = {Grid Research for Good (GRG) schema documentation, version 4.0} } - IA 2018Distributed multi-agent optimization for smart grids and home automationBest AI*IA Dissertation in AIFerdinando Fioretto, Agostino Dovier, and Enrico Pontelli
Distributed Constraint Optimization Problems (DCOPs) have emerged as one of the prominent multi-agent architectures to govern the agents’ autonomous behavior in a cooperative multi-agent system (MAS) where several agents coordinate with each other to optimize a global cost function taking into account their local preferences. They represent a powerful approach to the description and resolution of many practical problems. However, typical MAS applications are characterized by complex dynamics and interactions among a large number of entities, which translate into hard combinatorial problems, posing significant challenges from a computational and coordination standpoints. This paper reviews two methods to promote a hierarchical parallel model for solving DCOPs, with the aim of improving the performance of the DCOP algorithm. The first is a Multi-Variable Agent (MVA) DCOP decomposition, which exploits co-locality of an agent’s variables allowing the adoption of efficient centralized techniques to solve the subproblem of an agent. The second is the use of Graphics Processing Units (GPUs) to speed up a class of DCOP algorithms. Finally, exploiting these hierarchical parallel model, the paper presents two critical applications of DCOPs for demand response (DR) program in smart grids. The Multi-agent Economic Dispatch with Demand Response (EDDR), which provides an integrated approach to the economic dispatch and the DR model for power systems, and the Smart Home Device Scheduling (SHDS) problem, that formalizes the device scheduling and coordination problem across multiple smart homes to reduce energy peaks.
@article{FDP:ia19, title = {Distributed multi-agent optimization for smart grids and home automation}, author = {Fioretto, Ferdinando and Dovier, Agostino and Pontelli, Enrico}, journal = {Intelligenza Artificiale}, doi = {10.3233/IA-180037}, volume = {12}, number = {2}, pages = {67--87}, year = {2018}, url = {https://hdl.handle.net/11390/1144942}, } - JAIR 2018Distributed Constraint Optimization Problems and Applications: A SurveyFerdinando Fioretto, Enrico Pontelli, and William Yeoh
The field of Multi-Agent System (MAS) is an active area of research within Artificial Intelligence, with an increasingly important impact in industrial and other real-world applications. Within a MAS, autonomous agents interact to pursue personal interests and/or to achieve common objectives. Distributed Constraint Optimization Problems (DCOPs) have emerged as one of the prominent agent architectures to govern the agents’ autonomous behavior, where both algorithms and communication models are driven by the structure of the specific problem. During the last decade, several extensions to the DCOP model have enabled them to support MAS in complex, real-time, and uncertain environments. This survey aims at providing an overview of the DCOP model, giving a classification of its multiple extensions and addressing both resolution methods and applications that find a natural mapping within each class of DCOPs. The proposed classification suggests several future perspectives for DCOP extensions, and identifies challenges in the design of efficient resolution algorithms, possibly through the adaptation of strategies from different areas.
@article{FPY:jair18, author = {Fioretto, Ferdinando and Pontelli, Enrico and Yeoh, William}, title = {Distributed Constraint Optimization Problems and Applications: {A} Survey}, journal = {Journal of Artificial Intelligence Research}, volume = {61}, pages = {623--698}, year = {2018}, url = {https://doi.org/10.1613/jair.5565}, doi = {10.1613/jair.5565}, } - Constraints 2018Accelerating exact and approximate inference for (distributed) discrete optimization with GPUsFerdinando Fioretto, Enrico Pontelli, William Yeoh, and Rina Dechter
Discrete optimization is a central problem in artificial intelligence. The optimization of the aggregated cost of a network of cost functions arises in a variety of problems including Weighted Constraint Programs (WCSPs), Distributed Constraint Optimization (DCOP), as well as optimization in stochastic variants such as the tasks of finding the most probable explanation (MPE) in belief networks. Inference-based algorithms are powerful techniques for solving discrete optimization problems, which can be used independently or in combination with other techniques. However, their applicability is often limited by their compute intensive nature and their space requirements. This paper proposes the design and implementation of a novel inference-based technique, which exploits modern massively parallel architectures, such as those found in Graphical Processing Units (GPUs), to speed up the resolution of exact and approximated inference-based algorithms for discrete optimization. The paper studies the proposed algorithm in both centralized and distributed optimization contexts. The paper demonstrates that the use of GPUs provides significant advantages in terms of runtime and scalability, achieving up to two orders of magnitude in speedups and showing a considerable reduction in execution time (up to 345 times faster) with respect to a sequential version.
@article{FPYD:constraints18, author = {Fioretto, Ferdinando and Pontelli, Enrico and Yeoh, William and Dechter, Rina}, title = {Accelerating exact and approximate inference for (distributed) discrete optimization with GPUs}, journal = {Constraints - An International Journal}, volume = {23}, number = {1}, pages = {1--43}, year = {2018}, url = {https://doi.org/10.1007/s10601-017-9274-1}, doi = {10.1007/s10601-017-9274-1}, } - AI Matters 2018Ferdinando Fioretto, and William Yeoh
@article{FY:aimatters18, author = {Fioretto, Ferdinando and Yeoh, William}, title = {{AI} buzzwords explained: distributed constraint optimization problems}, journal = {{AI} Matters}, volume = {3}, number = {4}, pages = {8--13}, year = {2018}, url = {https://doi.org/10.1145/3175502.3175506}, doi = {10.1145/3175502.3175506}, } - TPLP 2018Ferdinando Fioretto, and Enrico Pontelli
@article{FP:tpc18, author = {Fioretto, Ferdinando and Pontelli, Enrico}, title = {Past and present (and future) of parallel and distributed computation in (constraint) logic programming}, journal = {Theory Pract. Log. Program.}, volume = {18}, number = {5-6}, pages = {722--724}, year = {2018}, url = {https://doi.org/10.1017/S1471068418000406}, doi = {10.1017/S1471068418000406}, } - AAMAS 2018Constrained-Based Differential Privacy for Mobility ServicesFerdinando Fioretto, Chansoo Lee, and Pascal Van Hentenryck
Ubiquitous mobile and wireless communication systems have the potential to revolutionize transportation systems, making accurate mobility traces and activity-based patterns available to optimize their design and operations. It also poses significant privacy risks, potentially revealing highly sensitive personal information. This paper studies the use of differential privacy to release mobility data that can be used for smart transportation systems. It shows that existing approaches do not provide the desired fidelity for practical uses. To remedy this critical limitation, the paper proposes the idea of optimization-based differential privacy that casts the production of a private dataset as an optimization problem that minimizes the impact of added Laplacian noise on the algorithmic task at hand. When applied to a city-level multi-modal transit system, experimental results show that the design and operations of the transportation system have similar performance measures when optimized over the real and private datasets. The results also indicate that optimization-based differential privacy may improve the accuracy of state-of-art privacy methods by an order of magnitude.
@inproceedings{FLvH:aamas18, author = {Fioretto, Ferdinando and Lee, Chansoo and Hentenryck, Pascal Van}, title = {Constrained-Based Differential Privacy for Mobility Services}, booktitle = {International Conference on Autonomous Agents and Multiagent Systems}, pages = {1405--1413}, publisher = {International Foundation for Autonomous Agents and Multiagent Systems Richland, SC, {USA} / {ACM}}, year = {2018}, } - CP 2018A Large Neighboring Search Schema for Multi-agent OptimizationKhoi D. Hoang, Ferdinando Fioretto, William Yeoh, Enrico Pontelli, and Roie ZivanIn International Conference on Principles and Practice of Constraint Programming, 2018|Abs|Bib|Online|AlphaXiv
The Distributed Constraint Optimization Problem (DCOP) is an elegant paradigm for modeling and solving multi-agent problems which are distributed in nature, and where agents cooperate to optimize a global objective within the confines of localized communication. Since solving DCOPs optimally is NP-hard, recent effort in the development of DCOP algorithms has focused on incomplete methods. Unfortunately, many of such proposals do not provide quality guarantees or provide a loose quality assessment. Thus, this paper proposes the Distributed Large Neighborhood Search (DLNS), a novel iterative local search framework to solve DCOPs, which provides guarantees on solution quality refining lower and upper bounds in an iterative process. Our experimental analysis of DCOP benchmarks on several important classes of graphs illustrates the effectiveness of DLNS in finding good solutions and tight upper bounds in both problems with and without hard constraints.
@inproceedings{HFWPZ:cp18, author = {Hoang, Khoi D. and Fioretto, Ferdinando and Yeoh, William and Pontelli, Enrico and Zivan, Roie}, title = {A Large Neighboring Search Schema for Multi-agent Optimization}, booktitle = {International Conference on Principles and Practice of Constraint Programming}, series = {Lecture Notes in Computer Science}, volume = {11008}, pages = {688--706}, publisher = {Springer}, year = {2018}, url = {https://doi.org/10.1007/978-3-319-98334-9_44}, doi = {10.1007/978-3-319-98334-9_44}, } - CPAIOR 2018Constrained-Based Differential Privacy: Releasing Optimal Power Flow Benchmarks Privately - Releasing Optimal Power Flow Benchmarks PrivatelyFerdinando Fioretto, and Pascal Van HentenryckIn Integration of Constraint Programming, Artificial Intelligence, and Operations Research, 2018|Abs|Bib|Online
This paper considers the problem of releasing optimal power flow benchmarks that maintain the privacy of customers (loads) using the notion of Differential Privacy. It is motivated by the observation that traditional differential-privacy mechanisms are not accurate enough: The dded noise fundamentally changes the nature of the underlying optimization and often leads to test cases with no solution. To remedy this limitation, the paper introduces the framework of Constraint-Based Differential Privacy (CBDP) that leverages the post-processing immunity of differential privacy to improve the accuracy of traditional mechanisms. More precisely, CBDP solves an optimization problem to satisfies the problem-specific constraints by redistributing the noise. The paper shows that CBDP enjoys desirable theoretical properties and produces orders of magnitude improvements on the largest set of test cases available.
@inproceedings{FvH:cpaior18, author = {Fioretto, Ferdinando and Hentenryck, Pascal Van}, title = {Constrained-Based Differential Privacy: Releasing Optimal Power Flow Benchmarks Privately - Releasing Optimal Power Flow Benchmarks Privately}, booktitle = {Integration of Constraint Programming, Artificial Intelligence, and Operations Research}, series = {Lecture Notes in Computer Science}, volume = {10848}, pages = {215--231}, publisher = {Springer}, year = {2018}, url = {https://doi.org/10.1007/978-3-319-93031-2_15}, doi = {10.1007/978-3-319-93031-2_15}, } - ISAIM 2018Constraint Composite Graph-Based Lifted Message Passing for Distributed Constraint Optimization ProblemsFerdinando Fioretto, Hong Xu, Sven Koenig, and T. K. Satish KumarIn International Symposium on Artificial Intelligence and Mathematics, ISAIM 2018, Fort Lauderdale, Florida, USA, January 3-5, 2018, 2018|Abs|Bib|Online
The Distributed Constraint Optimization Problem (DCOP) offers a powerful approach for the description and resolution of cooperative multi-agent problems. In this model, a group of agents coordinates their actions to optimize a global objective function, taking into account their local preferences. In the majority of DCOP algorithms, agents operate on three main graphical representations of the problem: (a) the constraint graph, (b) the pseudo-tree, or (c) the factor graph. In this paper, we introduce the Constraint Composite Graph (CCG) for DCOPs, an alternative graphical representation on which agents can coordinate their assignments to solve the distributed problem suboptimally. By leveraging this repre- sentation, agents are able to reduce the size of the problem. We propose a novel variant of Max-Sum–a popular DCOP incomplete algorithm–called CCG-Max-Sum, which is applied to CCGs. We also demonstrate the efficiency and effective- ness of CCG-Max-Sum on DCOP benchmarks based on several network topologies.
@inproceedings{FKK:isiam18, author = {Fioretto, Ferdinando and Xu, Hong and Koenig, Sven and Kumar, T. K. Satish}, title = {Constraint Composite Graph-Based Lifted Message Passing for Distributed Constraint Optimization Problems}, booktitle = {International Symposium on Artificial Intelligence and Mathematics, {ISAIM} 2018, Fort Lauderdale, Florida, USA, January 3-5, 2018}, year = {2018}, } - PRIMA 2018Solving Multiagent Constraint Optimization Problems on the Constraint Composite GraphFerdinando Fioretto, Hong Xu, Sven Koenig, and T. K. Satish Kumar
We introduce the Constraint Composite Graph (CCG) for Distributed Constraint Optimization Problems (DCOPs), a popular paradigm used for the description and resolution of cooperative multi-agent problems. The CCG is a novel graphical representation of DCOPs on which agents can coordinate their assignments to solve the distributed problem suboptimally. By leveraging this representation, agents are able to reduce the size of the problem. We propose a novel variant of Max- Sum—a popular DCOP incomplete algorithm—called CCG-Max-Sum, which is applied to CCGs, and demonstrate its efficiency and effectiveness on DCOP benchmarks based on several network topologies.
@inproceedings{FHKK:prima18, author = {Fioretto, Ferdinando and Xu, Hong and Koenig, Sven and Kumar, T. K. Satish}, title = {Solving Multiagent Constraint Optimization Problems on the Constraint Composite Graph}, booktitle = {Principles and Practice of Multi-Agent Systems}, series = {Lecture Notes in Computer Science}, volume = {11224}, pages = {106--122}, publisher = {Springer}, year = {2018}, url = {https://doi.org/10.1007/978-3-030-03098-8_7}, doi = {10.1007/978-3-030-03098-8_7}, }
2017
- AMEC 2017Investigation of Learning Strategies for the SPOT Broker in Power TACPorag Chowdhury, Russell Y. Folk, Ferdinando Fioretto, Christopher Kiekintveld, and William YeohAgent-Mediated Electronic Commerce. Designing Trading Strategies and Mechanisms for Electronic Markets (AMEC/TADA), 2017|Abs
The Power TAC simulation emphasizes the strategic problems that broker agents face in managing the economics of a smart grid. The brokers must make trades in multiple markets and, to be successful, brokers must make many good predictions about future supply, demand, and prices in the wholesale and tariff markets. In this paper, we investigate the feasibility of using learning strategies to improve the performance of our broker, SPOT. Specifically, we investigate the use of decision trees and neural networks to predict the clearing price in the wholesale market and the use of reinforcement learning to learn good strategies for pricing our tariffs in the tariff market. Our preliminary results show that our learning strategies are promising ways to improve the performance of the agent for future competitions.
- AAMAS 2017A Realistic Dataset for the Smart Home Device Scheduling Problem for DCOPsVisionary Workshop Paper AwardWilliam Kluegel, Muhammad A. Iqbal, Ferdinando Fioretto, William Yeoh, and Enrico Pontelli
The field of Distributed Constraint Optimization has gained momentum in recent years thanks to its ability to address various applications related to multi-agent cooperation. While techniques for solving Distributed Constraint Optimization Problems (DCOPs) are abundant and have matured substantially since the field inception, the number of DCOP realistic applications available to assess the performance of DCOP algorithms is lagging behind. To contrast this background we (i) introduce the Smart Home Device Scheduling (SHDS) problem, which describe the problem of coordinating smart devices schedules across multiple homes as a multi-agent system, (ii) detail the physical models adopted to simulate smart sensors, smart actuators, and homes’ environments, and (iii) introduce a DCOP realistic benchmark for SHDS problems.
@inproceedings{KF0P:aamas17, author = {Kluegel, William and Iqbal, Muhammad A. and Fioretto, Ferdinando and Yeoh, William and Pontelli, Enrico}, title = {A Realistic Dataset for the Smart Home Device Scheduling Problem for DCOPs}, booktitle = {{AAMAS} 2017 Workshops, Visionary Papers, Revised Selected Papers}, series = {Lecture Notes in Computer Science}, volume = {10643}, pages = {125--142}, publisher = {Springer}, year = {2017}, url = {https://doi.org/10.1007/978-3-319-71679-4_9}, doi = {10.1007/978-3-319-71679-4_9}, } - AAMAS 2017Infinite-Horizon Proactive Dynamic DCOPsKhoi D. Hoang, Ping Hou, Ferdinando Fioretto, William Yeoh, Roie Zivan, and Makoto Yokoo
The Distributed Constraint Optimization Problem (DCOP) formulation is a powerful tool for modeling multi-agent coordination problems. Researchers have recently extended this model to Proactive Dynamic DCOPs (PD-DCOPs) to capture the inherent dynamism present in many coordination problems. The PD-DCOP formulation is a finite-horizon model that assumes a finite horizon is known a priori. It ignores changes to the problem after the horizon and is thus not guaranteed to find optimal solutions for infinite-horizon problems, which often occur in the real world. Therefore, we (i) propose the Infinite-Horizon PD-DCOP (IPD- DCOP) model, which extends PD-DCOPs to handle infinite horizons; (ii) exploit the convergence properties of Markov chains to determine the optimal solution to the problem after it has converged; (iii) propose three distributed greedy algorithms to solve IPD-DCOPs; (iv) provide theoretical quality guarantees on the new model; and (v) empirically evaluate both proactive and reactive algorithms to determine the tradeoffs between the two classes. The final contribution is important as, thus far, researchers have exclusively evaluated the two classes of algorithms in isolation. As a result, it is difficult to identify the characteristics of problems that they excel in. Our results are the first in this important direction.
@inproceedings{HHF0ZY:aamas17, author = {Hoang, Khoi D. and Hou, Ping and Fioretto, Ferdinando and Yeoh, William and Zivan, Roie and Yokoo, Makoto}, title = {Infinite-Horizon Proactive Dynamic DCOPs}, booktitle = {International Conference on Autonomous Agents and Multiagent Systems}, pages = {212--220}, publisher = {{ACM}}, year = {2017}, } - AAMAS 2017A Multiagent System Approach to Scheduling Devices in Smart HomesFerdinando Fioretto, William Yeoh, and Enrico Pontelli
Demand-side management (DSM) in the smart grid allows customers to make autonomous decisions on their energy consumption, helping energy providers to reduce the energy peaks in load demand. The automated scheduling of smart devices in residential and commercial buildings plays a key role in DSM. Due to data privacy and user autonomy, such an approach is best implemented through distributed multi-agent systems. This paper makes the following contributions: (i) It introduces the Smart Home Device Scheduling (SHDS) problem, which formalizes the device scheduling and coordination problem across multiple smart homes as a multi-agent system; (ii) It describes a mapping of this problem to a distributed constraint optimization problem; (iii) It proposes a distributed algorithm for the SHDS problem; and (iv) It presents empirical results from a physically distributed system of Raspberry Pis, each capable of controlling smart devices through hardware interfaces, as well as larger scale synthetic experiments.
@inproceedings{F0P:aamas17, author = {Fioretto, Ferdinando and Yeoh, William and Pontelli, Enrico}, title = {A Multiagent System Approach to Scheduling Devices in Smart Homes}, booktitle = {International Conference on Autonomous Agents and Multiagent Systems}, pages = {981--989}, publisher = {{ACM}}, year = {2017}, } - AAMAS 2017A Distributed Constraint Optimization (DCOP) Approach to the Economic Dispatch with Demand ResponseFerdinando Fioretto, William Yeoh, Enrico Pontelli, Ye Ma, and Satishkumar J. Ranade
With the growing complexity of the current power grid, there is an increasing need for intelligent operations coordinating energy supply and demand. A key feature of the smart grid vision is that intelligent mechanisms will coordinate the production, transmission, and consumption of energy in a distributed and reliable way. Economic Dispatch (ED) and Demand Response (DR) are two key problems that need to be solved to achieve this vision. In traditional operations, ED and DR are implemented separately, despite the strong inter-dependencies between these two problems. Therefore, we propose an integrated approach to solve the ED and DR problems that simultaneously maximizes the benefits of customers and minimizes the generation costs, and introduce an effective multi-agent-based algorithm, based on Distributed Constraint Optimization Problems (DCOPs), acting on direct control of both generators and dispatchable loads. To cope with the high complexity of the problem, our solution employs General Purpose Graphical Processing Units (GPGPUs) to speed up the computational runtime. We empirically evaluate the proposed algorithms on standard IEEE bus systems and test the stability of the proposed solution with a state-of-the-art power system simulator on the IEEE 30-bus system.
@inproceedings{F0PMR:aamas17, author = {Fioretto, Ferdinando and Yeoh, William and Pontelli, Enrico and Ma, Ye and Ranade, Satishkumar J.}, title = {A Distributed Constraint Optimization {(DCOP)} Approach to the Economic Dispatch with Demand Response}, booktitle = {International Conference on Autonomous Agents and Multiagent Systems}, pages = {999--1007}, publisher = {{ACM}}, year = {2017}, } - CP 2017Preference Elicitation for DCOPsAtena M. Tabakhi, Tiep Le, Ferdinando Fioretto, and William YeohIn International Conference on Principles and Practice of Constraint Programming, 2017|Abs|Bib|Online
Distributed Constraint Optimization Problems (DCOPs) offer a powerful approach for the description and resolution of cooperative multi-agent problems. In this model, a group of agents coordinate their actions to optimize a global objective function, taking into account their preferences or constraints. A core limitation of this model is the assumption that the preferences of all agents or the costs of all constraints are specified a priori. Unfortunately, this assumption does not hold in a number of application domains where preferences or constraints must be elicited from the users. One of such domains is the Smart Home Device Scheduling (SHDS) problem. Motivated by this limitation, we make the following contributions in this paper: (1) We propose a general model for preference elicitation in DCOPs; (2) We propose several heuristics to elicit preferences in DCOPs; and (3) We empirically evaluate the effect of these heuristics on random binary DCOPs as well as SHDS problems.
@inproceedings{TLFY:cp17, author = {Tabakhi, Atena M. and Le, Tiep and Fioretto, Ferdinando and Yeoh, William}, title = {Preference Elicitation for DCOPs}, booktitle = {International Conference on Principles and Practice of Constraint Programming}, series = {Lecture Notes in Computer Science}, volume = {10416}, pages = {278--296}, publisher = {Springer}, year = {2017}, url = {https://doi.org/10.1007/978-3-319-66158-2_18}, doi = {10.1007/978-3-319-66158-2_18}, }
2016
- PhD Thesis 2016Ferdinando Fioretto
@phdthesis{DBLP:phd/it/Fioretto16, author = {Fioretto, Ferdinando}, title = {Exploiting the Structure of Distributed Constraint Optimization Problems}, school = {University of Udine, Italy}, year = {2016}, url = {https://hdl.handle.net/11390/1132725}, } - AMEC/TADA 2016Moinul Morshed Porag Chowdhury, Russell Y. Folk, Ferdinando Fioretto, Christopher Kiekintveld, and William YeohIn Agent-Mediated Electronic Commerce. Designing Trading Strategies and Mechanisms for Electronic Markets - AMEC/TADA Revised Selected Papers, 2016|Bib|Online
@inproceedings{CFFK0:tada16, author = {Chowdhury, Moinul Morshed Porag and Folk, Russell Y. and Fioretto, Ferdinando and Kiekintveld, Christopher and Yeoh, William}, title = {Investigation of Learning Strategies for the {SPOT} Broker in Power {TAC}}, booktitle = {Agent-Mediated Electronic Commerce. Designing Trading Strategies and Mechanisms for Electronic Markets - {AMEC/TADA} Revised Selected Papers}, series = {Lecture Notes in Business Information Processing}, volume = {271}, pages = {96--111}, publisher = {Springer}, year = {2016}, url = {https://doi.org/10.1007/978-3-319-54229-4_7}, doi = {10.1007/978-3-319-54229-4_7}, } - AAAI 2016Multi-Variable Agents Decomposition for DCOPsFerdinando Fioretto, William Yeoh, and Enrico Pontelli
The application of DCOP models to large problems faces two main limitations: (i) Modeling limitations, as each agent can handle only a single variable of the problem; and (ii) Resolution limitations, as current approaches do not exploit the local problem structure within each agent. This paper proposes a novel Multi-Variable Agent (MVA) DCOP decomposition technique, which: (i) Exploits the co-locality of each agent’s variables, allowing us to adopt efficient centralized techniques within each agent; (ii) Enables the use of hierarchical parallel models and proposes the use of GPUs; and (iii) Reduces the amount of computation and communication required in several classes of DCOP algorithms.
@inproceedings{F0P:aaai16, author = {Fioretto, Ferdinando and Yeoh, William and Pontelli, Enrico}, title = {Multi-Variable Agents Decomposition for DCOPs}, booktitle = {AAAI Conference on Artificial Intelligence}, pages = {2480--2486}, publisher = {{AAAI} Press}, year = {2016}, url = {http://www.aaai.org/ocs/index.php/AAAI/AAAI16/paper/view/12093}, } - AAMAS 2016Proactive Dynamic Distributed Constraint OptimizationKhoi D. Hoang, Ferdinando Fioretto, Ping Hou, Makoto Yokoo, William Yeoh, and Roie Zivan
Current approaches that model dynamism in DCOPs solve a sequence of static problems, reacting to changes in the environment as the agents observe them. Such approaches thus ignore possible predictions on future changes. To overcome this limitation, we introduce Proactive Dynamic DCOPs (PD-DCOPs), a novel formalism to model dynamic DCOPs in the presence of exogenous uncertainty. In contrast to reactive approaches, PD-DCOPs are able to explicitly model the possible changes to the problem, and take such information into account proactively, when solving the dynamically changing problem. The additional expressivity of this formalism allows it to model a wider variety of distributed optimization problems. Our work presents both theoretical and practical contributions that advance current dynamic DCOP models: (i) we introduce the PD-DCOP model, which explicitly captures dynamic changes of the DCOP over time; (ii) we discuss the complexity of this new class of DCOPs; and (iii) we develop both exact and approximation algorithms with quality guarantees to solve PD-DCOPs proactively.
@inproceedings{HFHY0Z:aamas16, author = {Hoang, Khoi D. and Fioretto, Ferdinando and Hou, Ping and Yokoo, Makoto and Yeoh, William and Zivan, Roie}, title = {Proactive Dynamic Distributed Constraint Optimization}, booktitle = {International Conference on Autonomous Agents and Multiagent Systems}, pages = {597--605}, publisher = {{ACM}}, year = {2016}, } - AAMAS 2016ER-DCOPs: A Framework for Distributed Constraint Optimization with Uncertainty in Constraint UtilitiesTiep Le, Ferdinando Fioretto, William Yeoh, Tran Cao Son, and Enrico Pontelli
Distributed Constraint Optimization Problems (DCOPs) have been used to model a number of multi-agent coordination problems. In DCOPs, agents are assumed to have complete information about the utility of their possible actions. However, in many real-world applications, such utilities are stochastic due to the presence of exogenous events that are beyond the direct control of the agents. This paper addresses this issue by extending the standard DCOP model to Expected Regret DCOP (ER-DCOP) for DCOP applications with uncertainty in constraint utilities. Different from other approaches, ER-DCOPs aim at minimizing the overall expected regret of the problem. The paper proposes the ER-DPOP algorithm for solving ER-DCOPs, which is complete and requires a linear number of messages with respect to the number of agents in the problem. We further present two implementations of ER-DPOP—GPU- and ASP-based implementations—that orthogonally exploit the problem structure and present their evaluations on random networks and power network problems.
@inproceedings{LeF0SP:aamas16, author = {Le, Tiep and Fioretto, Ferdinando and Yeoh, William and Son, Tran Cao and Pontelli, Enrico}, title = {ER-DCOPs: {A} Framework for Distributed Constraint Optimization with Uncertainty in Constraint Utilities}, booktitle = {International Conference on Autonomous Agents and Multiagent Systems}, pages = {606--614}, publisher = {{ACM}}, year = {2016}, } - CP 2016A Dynamic Programming-Based MCMC Framework for Solving DCOPs with GPUsFerdinando Fioretto, William Yeoh, and Enrico PontelliIn International Conference on Principles and Practice of Constraint Programming, 2016|Abs|Bib|Online
The field of Distributed Constraint Optimization (DCOP) has gained momentum in recent years, thanks to its ability to address various applications related to multi-agent coordination. Nevertheless, solving DCOPs is computationally challenging. Thus, in large scale, complex applications, incomplete DCOP algorithms are necessary. Recently, researchers have introduced a promising class of incomplete DCOP algorithms, based on sampling. However, this paradigm requires a multitude of samples to ensure convergence. This paper exploits the property that sampling is amenable to parallelization, and introduces a general framework, called Distributed MCMC (DMCMC), that is based on a dynamic programming procedure and uses Markov Chain Monte Carlo (MCMC) sampling algorithms to solve DCOPs. Additionally, DMCMC harnesses the parallel computing power of Graphical Processing Units (GPUs) to speed-up the sampling process. The experimental results show that DMCMC can find good solutions up to two order of magnitude faster than other incomplete DCOP algorithms.
@inproceedings{F0P:cp16, author = {Fioretto, Ferdinando and Yeoh, William and Pontelli, Enrico}, title = {A Dynamic Programming-Based {MCMC} Framework for Solving DCOPs with GPUs}, booktitle = {International Conference on Principles and Practice of Constraint Programming}, series = {Lecture Notes in Computer Science}, volume = {9892}, pages = {813--831}, publisher = {Springer}, year = {2016}, url = {https://doi.org/10.1007/978-3-319-44953-1_51}, doi = {10.1007/978-3-319-44953-1_51}, } - AISGSB 2016Khoi D. Hoang, Ferdinando Fioretto, Ping Hou, Makoto Yokoo, William Yeoh, and Roie ZivanIn AI for Smart Grids and Smart Buildings, Papers from the 2016 AAAI Workshop, Phoenix, Arizona, USA, February 12, 2016, 2016|Bib|Online
@inproceedings{DBLP:conf/aaai/HoangFHY0Z16, author = {Hoang, Khoi D. and Fioretto, Ferdinando and Hou, Ping and Yokoo, Makoto and Yeoh, William and Zivan, Roie}, title = {Proactive Dynamic DCOPs}, booktitle = {{AI} for Smart Grids and Smart Buildings, Papers from the 2016 {AAAI} Workshop, Phoenix, Arizona, USA, February 12, 2016}, series = {{AAAI} Technical Report}, volume = {{WS-16-04}}, publisher = {{AAAI} Press}, year = {2016}, url = {https://auld.aaai.org/Library/Workshops/ws16-04.php}, }
2015
- UTEP/NMSU 2015Khoi Hoang Dai, Ferdinando Fioretto, Ping Hou, and William YeohIn 17th Joint UTEP/NMSU Workshop on Mathematics, Computer Science, and Computational Sciences, Nov 2015One-page workshop abstract|Bib|PDF|Online
@inproceedings{Fioretto:UTEP15, title = {Dynamic Distributed Constraint Optimization}, author = {Dai, Khoi Hoang and Fioretto, Ferdinando and Hou, Ping and Yeoh, William}, booktitle = {17th Joint UTEP/NMSU Workshop on Mathematics, Computer Science, and Computational Sciences}, year = {2015}, month = nov, pages = {1}, address = {El Paso, Texas}, } - TOMACS 2015Constrained Community-Based Gene Regulatory Network InferenceFerdinando Fioretto, Agostino Dovier, and Enrico Pontelli
The problem of gene regulatory network inference is a major concern of systems biology. In recent years, a novel methodology has gained momentum, called community network approach. Community networks integrate predictions from individual methods in a “metapredictor,” in order to compose the advantages of different methods and soften individual limitations. This article proposes a novel methodology to integrate prediction ensembles using constraint programming, a declarative modeling and problem solving paradigm. Constraint programming naturally allows the modeling of dependencies among components of the problem as constraints, facilitating the integration and use of different forms of knowledge. The new paradigm, referred to as constrained community network, uses constraints to capture properties of the regulatory networks (e.g., topological properties) and to guide the integration of knowledge derived from different families of network predictions. The article experimentally shows the potential of this approach: The addition of biological constraints can offer significant improvements in prediction accuracy.
@article{FDP:tomacs15, author = {Fioretto, Ferdinando and Dovier, Agostino and Pontelli, Enrico}, title = {Constrained Community-Based Gene Regulatory Network Inference}, journal = {{ACM} Transaction on Modeling and Computer Simulation}, volume = {25}, number = {2}, pages = {11:1--11:26}, year = {2015}, url = {https://doi.org/10.1145/2688909}, doi = {10.1145/2688909}, } - AAAI 2015Exploiting the Structure of Distributed Constraint Optimization ProblemsFerdinando Fioretto
In the proposed thesis, we study Distributed Constraint Optimization Problems (DCOPs), which are problems where several agents coordinate with each other to optimize a global cost function. The use of DCOPs has gained momentum, due to their capability of addressing complex and naturally distributed problems. A majority of the work in DCOP addresses the resolution problem by detaching the model from the resolution process, where they assume that each agent controls exclusively one variable of the problem (Burke et al. 2006). This assumption often is not reflected in the model specifications, and may lead to inefficient communication requirements. Another limitation of current DCOP resolution methods is their inability to capitalize on the presence of structural information, which may allow incoherent/unnecessary data to reticulate among the agents (Yokoo 2001). The purpose of the proposed dissertation is to study how to adapt and integrate insights gained from centralized solving techniques in order to enhance DCOP performance and scalability, enabling their use for the resolution of real-world complex problems. To do so, we hypothesize that one can exploit the DCOP structure in both problem modeling and problem resolution phases.
@inproceedings{F:aaai15, author = {Fioretto, Ferdinando}, title = {Exploiting the Structure of Distributed Constraint Optimization Problems}, booktitle = {AAAI Conference on Artificial Intelligence}, pages = {4233}, publisher = {{AAAI} Press}, year = {2015}, url = {http://www.aaai.org/ocs/index.php/AAAI/AAAI15/paper/view/9293}, } - AAMAS 2015Multi-Variable Agents Decomposition for DCOPs to Exploit Multi-Level ParallelismFerdinando Fioretto, William Yeoh, and Enrico Pontelli
Current DCOP algorithms suffer from a major limiting assumption—each agent can handle only a single variable of the problem—which limits their scalability. This paper proposes a novel Multi-Variable Agent (MVA) DCOP decomposition, which: (i) Exploits co-locality of an agent’s variables, allowing us to adopt efficient centralized techniques; (ii) Enables the use of hierarchical parallel models, such us those based on GPGPUs; and (iii) Empirically reduces the amount of communication required in several classes of DCOP algorithms. Experimental results show that our MVA decomposition outperforms non-decomposed DCOP algorithms, in terms of network load and scalability.
@inproceedings{DBLP:conf/atal/Fioretto0P15, author = {Fioretto, Ferdinando and Yeoh, William and Pontelli, Enrico}, title = {Multi-Variable Agents Decomposition for DCOPs to Exploit Multi-Level Parallelism}, booktitle = {International Conference on Autonomous Agents and Multiagent Systems}, pages = {1823--1824}, publisher = {{ACM}}, year = {2015}, } - AAMAS 2015Large Neighborhood Search with Quality Guarantees for Distributed Constraint Optimization ProblemsFerdinando Fioretto, Federico Campeotto, Agostino Dovier, Enrico Pontelli, and William Yeoh
This paper proposes Distributed Large Neighborhood Search (D-LNS), an incomplete DCOP algorithm that builds on the strengths of centralized LNS. D-LNS: (i) is anytime; (ii) provides guarantees on solution quality (upper and lower bounds); and (iii) can learn online the best neighborhood to explore. Experimental results show that D-LNS outperforms other incomplete DCOP algorithms in random and scale-free network instances.
@inproceedings{FCDP0:aamas15, author = {Fioretto, Ferdinando and Campeotto, Federico and Dovier, Agostino and Pontelli, Enrico and Yeoh, William}, title = {Large Neighborhood Search with Quality Guarantees for Distributed Constraint Optimization Problems}, booktitle = {International Conference on Autonomous Agents and Multiagent Systems}, pages = {1835--1836}, publisher = {{ACM}}, year = {2015}, } - AAMAS 2015Exploiting the Structure of Distributed Constraint Optimization ProblemsFerdinando Fioretto
In the proposed thesis, we study Distributed Constraint Optimization Problems (DCOPs), which are problems where several agents coordinate with each other to optimize a global cost function. The use of DCOPs has gained momentum, due to their capability of addressing complex and naturally distributed problems. A majority of the work in DCOP addresses the resolution problem by detaching the model from the resolution process, where they assume that each agent controls exclusively one variable of the problem (Burke et al. 2006). This assumption often is not reflected in the model specifications, and may lead to inefficient communication requirements. Another limitation of current DCOP resolution methods is their inability to capitalize on the presence of structural information, which may allow incoherent/unnecessary data to reticulate among the agents (Yokoo 2001). The purpose of the proposed dissertation is to study how to adapt and integrate insights gained from centralized solving techniques in order to enhance DCOP performance and scalability, enabling their use for the resolution of real-world complex problems. To do so, we hypothesize that one can exploit the DCOP structure in both problem modeling and problem resolution phases.
@inproceedings{F:aamas15, author = {Fioretto, Ferdinando}, title = {Exploiting the Structure of Distributed Constraint Optimization Problems}, booktitle = {International Conference on Autonomous Agents and Multiagent Systems}, pages = {2007--2008}, publisher = {{ACM}}, year = {2015}, } - CP 2015Exploiting GPUs in Solving (Distributed) Constraint Optimization Problems with Dynamic ProgrammingFerdinando Fioretto, Tiep Le, Enrico Pontelli, William Yeoh, and Tran Cao SonIn International Conference on Principles and Practice of Constraint Programming, 2015|Abs|Bib|Online
This paper proposes the design and implementation of a dynamic programming based algorithm for (distributed) constraint optimization, which exploits modern massively parallel architectures, such as those found in modern Graphical Processing Units (GPUs). The paper studies the proposed algorithm in both centralized and distributed optimization contexts. The experimental analysis, performed on unstructured and structured graphs, shows the advantages of employing GPUs, resulting in enhanced performances and scalability.
@inproceedings{FLP0S:cp15, author = {Fioretto, Ferdinando and Le, Tiep and Pontelli, Enrico and Yeoh, William and Son, Tran Cao}, editor = {Pesant, Gilles}, title = {Exploiting GPUs in Solving (Distributed) Constraint Optimization Problems with Dynamic Programming}, booktitle = {International Conference on Principles and Practice of Constraint Programming}, series = {Lecture Notes in Computer Science}, volume = {9255}, pages = {121--139}, publisher = {Springer}, year = {2015}, url = {https://doi.org/10.1007/978-3-319-23219-5_9}, doi = {10.1007/978-3-319-23219-5_9}, }
2014
- WCB 2014Federico Campeotto, Alessandro Dal Palù, Agostino Dovier, Ferdinando Fioretto, and Enrico Pontelli
@inproceedings{Fioretto:WCB14, title = {Experimenting with FIASCO for Protein Structure Prediction}, year = {2014}, url = {https://bioinf.dimi.uniud.it/publication/11390-1024950/}, author = {Campeotto, Federico and Dal Palù, Alessandro and Dovier, Agostino and Fioretto, Ferdinando and Pontelli, Enrico}, } - ParSearchOpt 2014Federico Campeotto, Alessandro Dal Palù, Agostino Dovier, Ferdinando Fioretto, and Enrico Pontelli
@inproceedings{Fioretto:ParSearchOpt14, title = {Towards a Complete Constraint Solver on GPU}, year = {2014}, url = {https://bioinf.dimi.uniud.it/publication/11390-1024946/}, author = {Campeotto, Federico and Dal Palù, Alessandro and Dovier, Agostino and Fioretto, Ferdinando and Pontelli, Enrico}, booktitle = {Workshop on Parallel Methods for Search and Optimization (ParSearchOpt)}, } - AAMAS 2014GD-GIBBS: a GPU-based sampling algorithm for solving distributed constraint optimization problemsFerdinando Fioretto, Federico Campeotto, Luca Da Rin Fioretto, William Yeoh, and Enrico Pontelli
Researchers have recently introduced a promising new class of Distributed Constraint Optimization Problem (DCOP) algorithms that is based on sampling. This paradigm is very amenable to parallelization since sampling algorithms require a lot of samples to ensure convergence, and the sampling process can be designed to be executed in parallel. This paper presents GPU-based D-Gibbs (GD-Gibbs), which extends the Distributed Gibbs (D-Gibbs) sampling algorithm and harnesses the power of parallel computation of GPUs to solve DCOPs. Experimental results show that GD-Gibbs is faster than several other benchmark algorithms on a distributed meeting scheduling problem.
@inproceedings{FCF0P:aamas14, author = {Fioretto, Ferdinando and Campeotto, Federico and Fioretto, Luca Da Rin and Yeoh, William and Pontelli, Enrico}, title = {{GD-GIBBS:} a GPU-based sampling algorithm for solving distributed constraint optimization problems}, booktitle = {AAMAS}, pages = {1339--1340}, publisher = {{IFAAMAS/ACM}}, year = {2014}, } - CP 2014Improving DPOP with Branch Consistency for Solving Distributed Constraint Optimization ProblemsFerdinando Fioretto, Tiep Le, William Yeoh, Enrico Pontelli, and Tran Cao Son
The DCOP model has gained momentum in recent years thanks to its ability to capture problems that are naturally distributed and cannot be realistically addressed in a centralized manner. Dynamic programming based techniques have been recognized to be among the most effective techniques for building complete DCOP solvers (e.g., DPOP). Unfortunately, they also suffer from a widely recognized drawback: their messages are exponential in size. Another limitation is that most current DCOP algorithms do not actively exploit hard constraints, which are common in many real problems. This paper addresses these two limitations by introducing an algorithm, called BrC-DPOP, that exploits arc consistency and a form of consistency that applies to paths in pseudo-trees to reduce the size of the messages. Experimental results shows that BrC-DPOP uses messages that are up to one order of magnitude smaller than DPOP, and that it can scale up well, being able to solve problems that its counterpart can not.
@inproceedings{FLYPS:cp14, author = {Fioretto, Ferdinando and Le, Tiep and Yeoh, William and Pontelli, Enrico and Son, Tran Cao}, title = {Improving {DPOP} with Branch Consistency for Solving Distributed Constraint Optimization Problems}, booktitle = {CP}, series = {Lecture Notes in Computer Science}, volume = {8656}, pages = {307--323}, publisher = {Springer}, year = {2014}, url = {https://doi.org/10.1007/978-3-319-10428-7_24}, doi = {10.1007/978-3-319-10428-7_24}, } - ECAI 2014A GPU Implementation of Large Neighborhood Search for Solving Constraint Optimization ProblemsFederico Campeotto, Agostino Dovier, Ferdinando Fioretto, and Enrico Pontelli
Constraint programming has gained prominence as an effective and declarative paradigm for modeling and solving complex combinatorial problems. Techniques based on local search have proved practical to solve real-world problems, providing a good compromise between optimality and efficiency. In spite of the natural presence of concurrency, there has been relatively limited effort to use novel massively parallel architectures, such as those found in modern Graphical Processing Units (GPUs), to speedup local search techniques in constraint programming. This paper describes a novel framework which exploits parallelism from a popular local search method (the Large Neighborhood Search method), using GPUs.
@inproceedings{CDFP:ecai14, author = {Campeotto, Federico and Dovier, Agostino and Fioretto, Ferdinando and Pontelli, Enrico}, title = {A {GPU} Implementation of Large Neighborhood Search for Solving Constraint Optimization Problems}, booktitle = {European Conference on Artificial Intelligence}, series = {Frontiers in Artificial Intelligence and Applications}, volume = {263}, pages = {189--194}, publisher = {{IOS} Press}, year = {2014}, url = {https://doi.org/10.3233/978-1-61499-419-0-189}, doi = {10.3233/978-1-61499-419-0-189}, } - PADL 2014Exploring the Use of GPUs in Constraint SolvingFederico Campeotto, Alessandro Dal Palù, Agostino Dovier, Ferdinando Fioretto, and Enrico Pontelli
This paper presents an experimental study aimed at assessing the feasibility of parallelizing constraint propagation—with particular focus on arc-consistency—using Graphical Processing Units (GPUs). GPUs support a form of data parallelism that appears to be suitable to the type of processing required to cycle through constraints and domain values during consistency checking and propagation. The paper illustrates an implementation of a constraint solver capable of hybrid propagations (i.e., alternating CPU and GPU), and demonstrates the potential for competitiveness against sequential implementations.
@inproceedings{CPDF:padlP14, author = {Campeotto, Federico and Pal{\`{u}}, Alessandro Dal and Dovier, Agostino and Fioretto, Ferdinando and Pontelli, Enrico}, title = {Exploring the Use of GPUs in Constraint Solving}, booktitle = {Practical Aspects of Declarative Languages}, series = {Lecture Notes in Computer Science}, volume = {8324}, pages = {152--167}, publisher = {Springer}, year = {2014}, url = {https://doi.org/10.1007/978-3-319-04132-2_11}, doi = {10.1007/978-3-319-04132-2_11}, }
2013
- JAIR 2013A Constraint Solver for Flexible Protein ModelFederico Campeotto, Alessandro Dal Palù, Agostino Dovier, Ferdinando Fioretto, and Enrico Pontelli
his paper proposes the formalization and implementation of a novel class of constraints aimed at modeling problems related to placement of multi-body systems in the 3-dimensional space. Each multi-body is a system composed of body elements, connected by joint relationships and constrained by geometric properties. The emphasis of this investigation is the use of multi-body systems to model native conformations of protein structures—where each body represents an entity of the protein (e.g., an amino acid, a small peptide) and the geometric constraints are related to the spatial properties of the composing atoms. The paper explores the use of the proposed class of constraints to support a variety of different structural analysis of proteins, such as loop modeling and structure prediction. The declarative nature of a constraint-based encoding provides elaboration tolerance and the ability to make use of any additional knowledge in the analysis studies. The filtering capabilities of the proposed constraints also allow to control the number of representative solutions that are withdrawn from the conformational space of the protein, by means of criteria driven by uniform distribution sampling principles. In this scenario it is possible to select the desired degree of precision and/or number of solutions. The filtering component automatically excludes configurations that violate the spatial and geometric properties of the composing multi-body system. The paper illustrates the implementation of a constraint solver based on the multi-body perspective and its empirical evaluation on protein structure analysis problems.
- CMSB 2013Constraint Programming in Community-Based Gene Regulatory Network InferenceBest Paper AwardFerdinando Fioretto, and Enrico Pontelli
Gene Regulatory Network (GRN) inference is a major objective of Systems Biology. The complexity of biological systems and the lack of adequate data have posed many challenges to the inference problem. Community networks integrate predictions from individual methods in a “meta predictor”, in order to compose the advantages of different methods and soften individual limitations. This paper proposes a novel methodology to integrate prediction ensembles using Constraint Programming, a declarative modeling paradigm, which allows the formulation of dependencies among components of the problem, enabling the integration of diverse forms of knowledge. The paper experimentally shows the potential of this method: the addition of biological constraints can offer improvements in the prediction accuracy, and the method shows promising results in assessing biological hypothesis using constraints.
2012
- CP 2012A Filtering Technique for Fragment Assembly-Based Proteins Loop Modeling with ConstraintsFederico Campeotto, Alessandro Dal Palù, Agostino Dovier, Ferdinando Fioretto, and Enrico PontelliIn International Conference on Principles and Practice of Constraint Programming, 2012|Abs|Bib|OnlineAlso appeared in: WCB (CP 2012)
Methods to predict the structure of a protein often rely on the knowledge of macro sub-structures and their exact or approximated relative positions in space. The parts connecting these sub-structures are called loops and, in general, they are characterized by a high degree of freedom. The modeling of loops is a critical problem in predicting protein conformations that are biologically realistic. This paper introduces a class of constraints that models a general multi-body system; we present a proof of NP-completeness and provide filtering techniques, inspired by inverse kinematics, that can drastically reduce the search space of potential conformations. The paper shows the application of the constraint in solving the protein loop modeling problem, based on fragments assembly.
@inproceedings{CPDFP:cp12, author = {Campeotto, Federico and Pal{\`{u}}, Alessandro Dal and Dovier, Agostino and Fioretto, Ferdinando and Pontelli, Enrico}, title = {A Filtering Technique for Fragment Assembly-Based Proteins Loop Modeling with Constraints}, booktitle = {International Conference on Principles and Practice of Constraint Programming}, pages = {850--866}, year = {2012}, url = {http://dx.doi.org/10.1007/978-3-642-33558-7_61}, doi = {10.1007/978-3-642-33558-7_61}, }
2011
- Thesis 2011Ferdinando Fioretto
@mastersthesis{Fioretto:MSc11, title = {A Constraint Programming Approach for the Analysis of Proteins Conformations Via Fragment Assembly}, year = {2011}, author = {Fioretto, Ferdinando}, school = {New Mexico State University}, note = {Master's thesis} } - WCB 2011Michael R. Best, Kabi Bhattarai, Federico Campeotto, Alessandro Dal Palù, Hung Dang, Agostino Dovier, Ferdinando Fioretto, Federico Fogolari, Tiep Le, and Enrico Pontelli
@inproceedings{Fioretto:WCB11, title = {Introducing FIASCO: Fragment-based Interactive Assembly for Protein Structure Prediction with Constraints}, year = {2011}, pages = {27--36}, url = {https://air.unipr.it/handle/11381/2396968}, author = {Best, Michael R. and Bhattarai, Kabi and Campeotto, Federico and Dal Palù, Alessandro and Dang, Hung and Dovier, Agostino and Fioretto, Ferdinando and Fogolari, Federico and Le, Tiep and Pontelli, Enrico}, }