Página 5 dos resultados de 1096 itens digitais encontrados em 0.003 segundos

## Levantamento GPR 4D sobre um derrame de óleo usado em transformadores de energia elétrica: um estudo controlado em laboratório; GPR 4D aquisition over a spill of oil used in eletrical energy tranformers : A controlled laboratory study.

Bertolla, Luciana
Fonte: Biblioteca Digitais de Teses e Dissertações da USP Publicador: Biblioteca Digitais de Teses e Dissertações da USP
Tipo: Dissertação de Mestrado Formato: application/pdf
Relevância na Pesquisa
17.154604%

## Electron Transfer Ion/Ion Reactions in a Three-Dimensional Quadrupole Ion Trap: Reactions of Doubly and Triply Protonated Peptides with SO2·−

Pitteri, Sharon J.; Chrisman, Paul A.; Hogan, Jason M.; McLuckey, Scott A.
Tipo: Artigo de Revista Científica
Relevância na Pesquisa
17.25886%
Ion–ion reactions between a variety of peptide cations (doubly and triply charged) and SO2 anions have been studied in a 3-D quadrupole ion trap, resulting in proton and electron transfer. Electron transfer dissociation (ETD) gives many c- and z-type fragments, resulting in extensive sequence coverage in the case of triply protonated peptides with SO2·−. For triply charged neurotensin, in which a direct comparison can be made between 3-D and linear ion trap results, abundances of ETD fragments relative to one another appear to be similar. Reactions of doubly protonated peptides with SO2·− give much less structural information from ETD than triply protonated peptides. Collision-induced dissociation (CID) of singly charged ions formed in reactions with SO2·− shows a combination of proton and electron transfer products. CID of the singly charged species gives more structural information than ETD of the doubly protonated peptide, but not as much information as ETD of the triply protonated peptide.

## A Proteomics Grade Electron Transfer Dissociation-enabled Hybrid Linear Ion Trap-orbitrap Mass Spectrometer

McAlister, Graeme C.; Berggren, W. Travis; Horning, Stevan; Makarov, Alexander; Phanstiel, Doug; Griep-Raming, Jens; Stafford, George; Swaney, Danielle L.; Syka, John E. P.; Zabrouskov, Vlad; Coon, Joshua J.
Tipo: Artigo de Revista Científica
Português
Relevância na Pesquisa
17.154604%
Here we describe the modification of a quadrupole linear ion trap-orbitrap hybrid (QLT-orbitrap) mass spectrometer to accommodate a negative chemical ionization (NCI) source. The NCI source is used to produce fluoranthene radical anions for imparting electron transfer dissociation (ETD). The anion beam is stable, robust, and intense so that a sufficient amount of reagents can be injected into the QLT in only 4 - 8 ms. Following ion/ion reaction in the QLT, ETD product ions are mass-to-charge (m/z) analyzed in either the QLT (for speed and sensitivity) or the orbitrap (for mass resolution and accuracy). Here we describe the physical layout of this device, parametric optimization of anion transport, an evaluation of relevant ETD figures of merit, and the application of this instrument to protein sequence analysis. Described proteomic applications include complex peptide mixture analysis, post-translational modification (PTM) site identification, isotope-encoded quantitation, large peptide characterization, and intact protein analysis. From these experiments we conclude the ETD-enabled orbitrap will provide the proteomic field with several new opportunities and represents an advance in protein sequence analysis technologies.

## Rapidly Alternating Transmission Mode Electron Transfer Dissociation and Collisional Activation for the Characterization of Polypeptide Ions

Han, Hongling; Xia, Yu; Yang, Min; McLuckey, Scott A.
Tipo: Artigo de Revista Científica
Português
Relevância na Pesquisa
17.25886%
Cation transmission/electron transfer reagent anion storage mode electron transfer ion/ion reactions and beam-type collisional activation of the polypeptide ions are performed in rapid succession in the high pressure collision cell (Q2) of a quadrupole/time-of-flight tandem mass spectrometer (QqTOF), where the electron transfer reagent anions are accumulated. Duty cycles for both electron transfer dissociation (ETD) and collision-induced dissociation (CID) experiments are improved relative to ion trapping approaches since there are no discrete ion storage and reaction steps for ETD experiments and no discrete ion storage step and frequency tuning for CID experiments. For this technique, moderately high resolution and mass accuracy are also obtained due to mass analysis via the TOF analyzer. This relatively simple approach has been demonstrated with a triply charged tryptic peptide, a triply charged tryptic phosphopeptide, and a triply charged tryptic N-linked glycopeptide. For the tryptic peptide, the sequence is identified with more certainty than would be available from a single method alone due to the complementary information provided by these two dissociation methods. Because of the complementary information derived from both ETD and CID dissociation methods...

## Peptide Quantification Using 8-Plex Isobaric Tags and Electron Transfer Dissociation Tandem Mass Spectrometry

Phanstiel, Doug; Unwin, Richard; McAlister, Graeme C.; Coon, Joshua J.
Tipo: Artigo de Revista Científica
Relevância na Pesquisa
17.154604%
Isobaric tags for absolute and relative quantitation (iTRAQ) allow for simultaneous relative quantification of peptides from up to eight different samples. Typically peptides labeled with 8-plex iTRAQ tags are pooled and fragmented using beam-type collision activated dissociation (CAD) which, in addition to cleaving the peptide backbone bonds, cleaves the tag to produce reporter ions. The relative intensities of the reporters are directly proportional to the relative abundances of each peptide in the solution phase. Recently, studies using the 4-plex iTRAQ tagging reagent demonstrated that electron transfer dissociation (ETD) of 4-plex iTRAQ labeled peptides cleaves at the N-Cα bond in the tag and allows for up to three channels of quantification. In this paper we investigate the ETD fragmentation patterns of peptides labeled with 8-plex iTRAQ tags. We demonstrate that upon ETD, peptides labeled with 8-plex iTRAQ tags fragment to produce unique reporter ions that allow for five channels of quantification. ETD-MS/MS of these labeled peptides also produces a peak at 322 m/z which, upon resonant excitation (CAD), gives rise to all eight iTRAQ reporter ions and allows for eight channels of quantification. Comparison of this method to beam-type CAD quantification shows a good correlation (y) = 0.91x + 0.01...

## Electron Transfer Dissociation of Peptides Generated by Microwave D-Cleavage Digestion of Proteins

Hauser, Nicolas J.; Basile, Franco; Han, Hongling; McLuckey, Scott A.
Tipo: Artigo de Revista Científica
Português
Relevância na Pesquisa
17.154604%
The non-enzymatic digestion of proteins by microwave D-cleavage is an effective technique for site-specific cleavage at aspartic acid (D). This specific cleavage C-terminal to D residues leads to inherently large peptides (15–25 amino acids) that are usually relatively highly charged (above +3) when ionized by electrospray ionization (ESI) due to the presence of several basic amino acids within their sequences. It is well documented that highly charged peptide ions generated by ESI are well-suited for electron transfer dissociation (ETD), which produces c- and z-type fragment ions via gas-phase ion/ion reactions. In this paper we describe the sequence analysis by ETD tandem mass spectrometry (MS/MS) of multiply charged peptides generated by microwave D-cleavage of several standard proteins. Results from ETD measurements are directly compared to CID MS/MS of the same multiply charged precursor ions. Our results demonstrate that the non-enzymatic microwave D-cleavage technique is a rapid (< 6 min) and specific alternative to enzymatic cleavage with Lys-C or Asp-N to produce highly charged peptides that are amenable to informative ETD.

## Elucidation of O-glycosylation structures of the ß-amyloid precursor protein by liquid chromatography - mass spectrometry using electron transfer dissociation and collision induced dissociation

Perdivara, Irina; Petrovich, Robert; Alliquant, Bernadette; Deterding, Leesa J.; Tomer, Kenneth B.; Przybylski, Michael
Tipo: Artigo de Revista Científica
Relevância na Pesquisa
17.154604%
Accumulation and deposition of ß-amyloid peptide, a major constituent in neuritic plaques are hallmarks of Alzheimer’s disease (AD) and AD-related neurodegenerative diseases. ß-Amyloid (Aß) is derived from the proteolytic cleavage of amyloid precursor protein (APP), a transmembrane protein present in three major isoforms in brain comprising 695, 751 and 770 amino acids, respectively. Among other post-translational modifications, APP is modified during maturation by N- and O- glycosylation, which are thought to be responsible for its expression and secretion. Unlike N-glycosylation, no sites of O-glycosylation of APP have previously been reported. We report here the identification of three specific O-glycosylation sites of the secreted APP695 (sAPP695) produced in CHO cells, using a combination of high performance liquid chromatography and electrospray - tandem mass spectrometry. With the use of electron transfer dissociation and collision induced dissociation (ETD and CID), we identified type, composition and structures of the Core 1 type O-linked glycans attached at the residues: Thr 291, Thr 292 and Thr 576 of the full length APP695. The glycosylations comprise multiple short glycans, containing N-acetyl galactosamine (GalNAc)...

## A Mixed Integer Linear Optimization Framework for the Identification and Quantification of Targeted Post-translational Modifications of Highly Modified Proteins Using Multiplexed Electron Transfer Dissociation Tandem Mass Spectrometry*

DiMaggio, Peter A.; Young, Nicolas L.; Baliban, Richard C.; Garcia, Benjamin A.; Floudas, Christodoulos A.
Fonte: The American Society for Biochemistry and Molecular Biology Publicador: The American Society for Biochemistry and Molecular Biology
Tipo: Artigo de Revista Científica
Português
Relevância na Pesquisa
17.25886%
Here we present a novel methodology for the identification of the targeted post-translational modifications present in highly modified proteins using mixed integer linear optimization and electron transfer dissociation (ETD) tandem mass spectrometry. For a given ETD tandem mass spectrum, the rigorous set of modified forms that satisfy the mass of the precursor ion, within some tolerance error, are enumerated by solving a feasibility problem via mixed integer linear optimization. The enumeration of the entire superset of modified forms enables the method to normalize the relative contributions of the individual modification sites. Given the entire set of modified forms, a superposition problem is then formulated using mixed integer linear optimization to determine the relative fractions of the modified forms that are present in the multiplexed ETD tandem mass spectrum. Chromatographic information in the mass and time dimension is utilized to assess the likelihood of the assigned modification states, to average several tandem mass spectra for confident identification of lower level forms, and to infer modification states of partially assigned spectra. The utility of the proposed computational framework is demonstrated on an entire LC-MS/MS ETD experiment corresponding to a mixture of highly modified histone peptides. This new computational method will facilitate the unprecedented LC-MS/MS ETD analysis of many hypermodified proteins and offer novel biological insight into these previously understudied systems.

## Combining High-energy C-trap Dissociation and Electron Transfer Dissociation for Protein O-GlcNAc Modification Site Assignment

Zhao, Peng; Viner, Rosa; Teo, Chin Fen; Boons, Geert-Jan; Horn, David; Wells, Lance
Tipo: Artigo de Revista Científica
Português
Relevância na Pesquisa
17.25886%
Mass spectrometry-based studies of proteins that are post-translationally modified by O-linked β-N-acetylglucosamine (O-GlcNAc) are challenged in effectively identifying the sites of modification while simultaneously sequencing the peptides. Here we tested the hypothesis that a combination of high-energy C-trap dissociation (HCD) and electron transfer dissociation (ETD) could specifically target the O-GlcNAc modified peptides and elucidate the amino acid sequence while preserving the attached GlcNAc residue for accurate site assignment. By taking advantage of the recently characterized O-GlcNAc-specific IgG monoclonal antibodies and the combination of HCD and ETD fragmentation techniques, O-GlcNAc modified proteins were enriched from HEK293T cells and subsequently characterized using the LTQ Orbitrap Velos™ ETD (Thermo Fisher Scientific) mass spectrometer. In our dataset, 83 sites of O-GlcNAc modification are reported with high confidence confirming that the HCD/ETD combined approach is amenable to the detection and site assignment of O-GlcNAc modified peptides. Realizing HCD triggered ETD fragmentation on a linear ion trap/Orbitrap platform for more in-depth analysis and application of this technique to other post-translationally modified proteins are currently underway. Furthermore...

## Optimization of Electron Transfer Dissociation via Informed Selection of Reagents and Operating Parameters

Compton, Philip D.; Strukl, Joseph V.; Bai, Dina L.; Shabanowitz, Jeffrey; Hunt, Donald F.
Tipo: Artigo de Revista Científica
Português
Relevância na Pesquisa
17.334155%
Electron transfer dissociation (ETD) has improved the mass spectrometric analysis of proteins and peptides with labile post-translational modifications and larger intact masses. Here, the parameters governing the reaction rate of ETD are examined experimentally. Currently, due to reagent injection and isolation events as well as longer reaction times, ETD spectra require significantly more time to acquire than collision-induced dissociation (CID) spectra (>100 ms), resulting in a trade-off in the dynamic range of tandem MS analyses when ETD-based methods are compared to CID-based methods. Through fine adjustment of reaction parameters and the selection of reagents with optimal characteristics, we demonstrate a drastic reduction in the time taken per ETD event. In fact, ETD can be performed with optimal efficiency in nearly the same time as CID at low precursor charge state (z = +3) and becomes faster at higher charge state (z > +3).

## Increasing the Productivity of Glycopeptides Analysis by Using Higher-Energy Collision Dissociation-Accurate Mass-Product-Dependent Electron Transfer Dissociation

Saba, Julian; Dutta, Sucharita; Hemenway, Eric; Viner, Rosa
Fonte: Hindawi Publishing Corporation Publicador: Hindawi Publishing Corporation
Tipo: Artigo de Revista Científica
Português
Relevância na Pesquisa
17.25886%
Currently, glycans are attracting attention from the scientific community as potential biomarkers or as posttranslational modifications (PTMs) of therapeutic proteins. However, structural characterization of glycoproteins and glycopeptides remains analytically challenging. Here, we report on the implementation of a novel acquisition strategy termed higher-energy collision dissociation-accurate mass-product-dependent electron transfer dissociation (HCD-PD-ETD) on a hybrid linear ion trap-orbitrap mass spectrometer. This acquisition strategy uses the complementary fragmentations of ETD and HCD for glycopeptides analysis in an intelligent fashion. Furthermore, the approach minimizes user input for optimizing instrumental parameters and enables straightforward detection of glycopeptides. ETD spectra are only acquired when glycan oxonium ions from MS/MS HCD are detected. The advantage of this approach is that it streamlines data analysis and improves dynamic range and duty cycle. Here, we present the benefits of HCD-PD-ETD relative to the traditional alternating HCD/ETD for a trainer set containing twelve-protein mixture with two glycoproteins: human serotransferrin, ovalbumin and contaminations of two other: bovine alpha 1 acid glycoprotein (bAGP) and bovine fetuin.

## Enhanced Peptide Identification by Electron Transfer Dissociation Using an Improved Mascot Percolator*

Wright, James C.; Collins, Mark O.; Yu, Lu; Käll, Lukas; Brosch, Markus; Choudhary, Jyoti S.
Fonte: The American Society for Biochemistry and Molecular Biology Publicador: The American Society for Biochemistry and Molecular Biology
Tipo: Artigo de Revista Científica
Português
Relevância na Pesquisa
17.334155%
Peptide identification using tandem mass spectrometry is a core technology in proteomics. Latest generations of mass spectrometry instruments enable the use of electron transfer dissociation (ETD) to complement collision induced dissociation (CID) for peptide fragmentation. However, a critical limitation to the use of ETD has been optimal database search software. Percolator is a post-search algorithm, which uses semi-supervised machine learning to improve the rate of peptide spectrum identifications (PSMs) together with providing reliable significance measures. We have previously interfaced the Mascot search engine with Percolator and demonstrated sensitivity and specificity benefits with CID data. Here, we report recent developments in the Mascot Percolator V2.0 software including an improved feature calculator and support for a wider range of ion series. The updated software is applied to the analysis of several CID and ETD fragmented peptide data sets. This version of Mascot Percolator increases the number of CID PSMs by up to 80% and ETD PSMs by up to 60% at a 0.01 q-value (1% false discovery rate) threshold over a standard Mascot search, notably recovering PSMs from high charge state precursor ions. The greatly increased number of PSMs and peptide coverage afforded by Mascot Percolator has enabled a fuller assessment of CID/ETD complementarity to be performed. Using a data set of CID and ETcaD spectral pairs...

## DRY–WET CYCLES INCREASE PESTICIDE RESIDUE RELEASE FROM SOIL

Jablonowski, Nicolai David; Linden, Andreas; Köppchen, Stephan; Thiele, Björn; Hofmann, Diana; Burauel, Peter
Fonte: John Wiley & Sons, Inc. Publicador: John Wiley & Sons, Inc.
Tipo: Artigo de Revista Científica
Português
Relevância na Pesquisa
17.25886%
Soil drying and rewetting may alter the release and availability of aged pesticide residues in soils. A laboratory experiment was conducted to evaluate the influence of soil drying and wetting on the release of pesticide residues. Soil containing environmentally long-term aged (9–17 years) 14C-labeled residues of the herbicides ethidimuron (ETD) and methabenzthiazuron (MBT) and the fungicide anilazine (ANI) showed a significantly higher release of 14C activity in water extracts of previously dried soil compared to constantly moistened soil throughout all samples (ETD: p < 0.1, MBT and ANI: p < 0.01). The extracted 14C activity accounted for 44% (ETD), 15% (MBT), and 20% (ANI) of total residual 14C activity in the samples after 20 successive dry–wet cycles, in contrast to 15% (ETD), 5% (MBT), and 6% (ANI) in extracts of constantly moistened soils. In the dry–wet soils, the dissolved organic carbon (DOC) content correlated with the measured 14C activity in the aqueous liquids and indicated a potential association of DOC with the pesticide molecules. Liquid chromatography MS/MS analyses of the water extracts of dry–wet soils revealed ETD and MBT in detectable amounts, accounting for 1.83 and 0.01%, respectively, of total applied water-extractable parent compound per soil layer. These findings demonstrate a potential remobilization of environmentally aged pesticide residue fractions from soils due to abiotic stresses such as wet–dry cycles. Environ. Toxicol. Chem. 2012; 31: 1941–1947. © 2012 SETAC

## Statistical Analysis of Electron Transfer Dissociation Pairwise Fragmentation Patterns

Li, Wenzhou; Song, Chi; Bailey, Derek J.; Tseng, George C.; Coon, Joshua J.; Wysocki, Vicki H.
Tipo: Artigo de Revista Científica
Português
Relevância na Pesquisa
17.334155%
Electron transfer dissociation (ETD) is an alternative peptide dissociation method developed in recent years. Compared with the traditional collision induced dissociation (CID) b and y ion formation, ETD generates c and z ions and the backbone cleavage is believed to be less selective. We have reported previously the application of a statistical data mining strategy, K-means clustering, to discover fragmentation patterns for CID, and here we report application of this approach to ETD spectra. We use ETD data sets from digestions with three different proteases. Data analysis shows that selective cleavages do exist for ETD, with the fragmentation patterns affected by protease, charge states, and amino acid residue compositions. It is also noticed that the cn-1 ion, corresponding to loss of the C-terminal amino acid residue, is statistically strong regardless of the residue at the C-terminus of the peptide, which suggest that peptide gas phase conformation plays important roles in the dissociation pathways. These patterns provide a basis for mechanism elucidation, spectral prediction, and improvement of ETD peptide identification algorithms.

## Glycoproteomic Analysis of the Secretome of Human Endothelial Cells*

Yin, Xiaoke; Bern, Marshall; Xing, Qiuru; Ho, Jenny; Viner, Rosa; Mayr, Manuel
Fonte: The American Society for Biochemistry and Molecular Biology Publicador: The American Society for Biochemistry and Molecular Biology
Tipo: Artigo de Revista Científica
Português
Relevância na Pesquisa
17.25886%
Previous proteomics studies have partially unraveled the complexity of endothelial protein secretion but have not investigated glycosylation, a key modification of secreted and membrane proteins for cell communication. In this study, human umbilical vein endothelial cells were kept in serum-free medium before activation by phorbol-12-myristate-13 acetate, a commonly used secretagogue that induces exocytosis of endothelial vesicles. In addition to 123 secreted proteins, the secretome was particularly rich in membrane proteins. Glycopeptides were enriched by zwitterionic hydrophilic interaction liquid chromatography resins and were either treated with PNGase F and H218O or directly analyzed using a recently developed workflow combining higher-energy C-trap dissociation (HCD) with electron-transfer dissociation (ETD) for a hybrid linear ion trap–orbitrap mass spectrometer. After deglycosylation with PNGase F in the presence of H218O, 123 unique peptides displayed 18O-deamidation of asparagine, corresponding to 86 proteins with a total of 121 glycosylation sites. Direct glycopeptide analysis via HCD-ETD identified 131 glycopeptides from 59 proteins and 118 glycosylation sites, of which 41 were known, 51 were predicted, and 26 were novel. Two methods were compared: alternating HCD-ETD and HCD-product-dependent ETD. The former detected predominantly high-intensity...

## Electron Transfer Dissociation Mass Spectrometry in Proteomics

Kim, Min-Sik; Pandey, Akhilesh
Tipo: Artigo de Revista Científica
Português
Relevância na Pesquisa
17.334155%
Mass spectrometry has rapidly evolved to become the platform of choice for proteomic analysis. While CID remains the major fragmentation method for peptide sequencing, electron transfer dissociation (ETD) is emerging as a complementary method for characterization of peptides and post-translational modifications (PTMs). Here, we review the evolution of ETD and some of its newer applications including characterization of PTMs, non-tryptic peptides and intact proteins. We will also discuss some of the unique features of ETD such as its complementarity with CID and the use of alternating CID/ETD along with issues pertaining to analysis of ETD data. The potential of ETD for applications such as multiple reaction monitoring and proteogenomics in the future will also be discussed.

## High-throughput Database Search and Large-scale Negative Polarity Liquid Chromatography–Tandem Mass Spectrometry with Ultraviolet Photodissociation for Complex Proteomic Samples*

Madsen, James A.; Xu, Hua; Robinson, Michelle R.; Horton, Andrew P.; Shaw, Jared B.; Giles, David K.; Kaoud, Tamer S.; Dalby, Kevin N.; Trent, M. Stephen; Brodbelt, Jennifer S.
Fonte: The American Society for Biochemistry and Molecular Biology Publicador: The American Society for Biochemistry and Molecular Biology
Tipo: Artigo de Revista Científica
Português
Relevância na Pesquisa
17.25886%
The use of ultraviolet photodissociation (UVPD) for the activation and dissociation of peptide anions is evaluated for broader coverage of the proteome. To facilitate interpretation and assignment of the resulting UVPD mass spectra of peptide anions, the MassMatrix database search algorithm was modified to allow automated analysis of negative polarity MS/MS spectra. The new UVPD algorithms were developed based on the MassMatrix database search engine by adding specific fragmentation pathways for UVPD. The new UVPD fragmentation pathways in MassMatrix were rigorously and statistically optimized using two large data sets with high mass accuracy and high mass resolution for both MS1 and MS2 data acquired on an Orbitrap mass spectrometer for complex Halobacterium and HeLa proteome samples. Negative mode UVPD led to the identification of 3663 and 2350 peptides for the Halo and HeLa tryptic digests, respectively, corresponding to 655 and 645 peptides that were unique when compared with electron transfer dissociation (ETD), higher energy collision-induced dissociation, and collision-induced dissociation results for the same digests analyzed in the positive mode. In sum, 805 and 619 proteins were identified via UVPD for the Halobacterium and HeLa samples...

## Characterizing O-linked glycopeptides by electron transfer dissociation: fragmentation rules and applications in data analysis

Zhu, Zhikai; Su, Xiaomeng; Clark, Daniel F.; Go, Eden P.; Desaire, Heather
Tipo: Artigo de Revista Científica
Português
Relevância na Pesquisa
17.334155%
Studying protein O-glycosylation remains an analytical challenge. Different from N-linked glycans, the O-glycosylation site is not within a known consensus sequence. Additionally, O-glycans are heterogeneous with numerous potential modification sites. Electron transfer dissociation (ETD) is the method of choice in analyzing these glycopeptides since the glycan side chain is intact in ETD, and the glycosylation site can be localized on the basis of the c and z fragment ions. Nonetheless, new software is necessary for interpreting O-glycopeptide ETD spectra in order to expedite the analysis workflow. To address the urgent need, we studied the fragmentation of O-glycopeptides in ETD and found useful rules that facilitate their identification. By implementing the rules into an algorithm to score potential assignments against ETD-MS/MS data, we applied the method to glycopeptides generated from various O-glycosylated proteins including mucin, erythropoietin, fetuin and an HIV envelope protein, 1086.C gp120. The site-specific O-glycopeptide composition was correctly assigned in every case, proving the merits of our method in analyzing glycopeptide ETD data. The algorithm described herein can be easily incorporated into other automated glycomics tools.

## Weak Convergence Properties of Constrained Emphatic Temporal-difference Learning with Constant and Slowly Diminishing Stepsize

Yu, Huizhen
Tipo: Artigo de Revista Científica
We consider the emphatic temporal-difference (TD) algorithm, ETD($\lambda$), for learning the value functions of stationary policies in a discounted, finite state and action Markov decision process. The ETD($\lambda$) algorithm was recently proposed by Sutton, Mahmood, and White to solve a long-standing divergence problem of the standard TD algorithm when it is applied to off-policy training, where data from an exploratory policy are used to evaluate other policies of interest. The almost sure convergence of ETD($\lambda$) has been proved in our recent work under general off-policy training conditions, but for a narrow range of diminishing stepsize. In this paper we present convergence results for constrained versions of ETD($\lambda$) with constant stepsize and with diminishing stepsize from a broad range. Our results characterize the asymptotic behavior of the trajectory of iterates produced by those algorithms, and are derived by combining key properties of ETD($\lambda$) with powerful convergence theorems from the weak convergence methods in stochastic approximation theory. For the case of constant stepsize, in addition to analyzing the behavior of the algorithms in the limit as the stepsize parameter approaches zero, we also analyze their behavior for a fixed stepsize and bound the deviations of their averaged iterates from the desired solution. These results are obtained by exploiting the weak Feller property of the Markov chains associated with the algorithms...
We consider the off-policy evaluation problem in Markov decision processes with function approximation. We propose a generalization of the recently introduced \emph{emphatic temporal differences} (ETD) algorithm \citep{SuttonMW15}, which encompasses the original ETD($\lambda$), as well as several other off-policy evaluation algorithms as special cases. We call this framework \ETD, where our introduced parameter $\beta$ controls the decay rate of an importance-sampling term. We study conditions under which the projected fixed-point equation underlying \ETD\ involves a contraction operator, allowing us to present the first asymptotic error bounds (bias) for \ETD. Our results show that the original ETD algorithm always involves a contraction operator, and its bias is bounded. Moreover, by controlling $\beta$, our proposed generalization allows trading-off bias for variance reduction, thereby achieving a lower total error.; Comment: arXiv admin note: text overlap with arXiv:1508.03411