Fhyujk

Cryptic with missing capitals

Can an insurance company drop you after receiving a bill and refusing to pay?

Why do members of Congress in committee hearings ask witnesses the same question multiple times?

Why does String.replaceAll() work differently in Java 8 from Java 9?

How to deal with an incendiary email that was recalled

Why zero tolerance on nudity in space?

Why exactly do action photographers need high fps burst cameras?

Slow moving projectiles from a hand-held weapon - how do they reach the target?

How to prevent cleaner from hanging my lock screen in Ubuntu 16.04

Lick explanation

"Free" Hopf algebra

Why does a metal block make a shrill sound but not a wooden block upon hammering?

Injecting creativity into a cookbook

Explain the objections to these measures against human trafficking

Why would the Pakistan airspace closure cancel flights not headed to Pakistan itself?

Breaking a Loop in Tikz

Is it a fallacy if someone claims they need an explanation for every word of your argument to the point where they don't understand common terms?

Strange Sign on Lab Door

Does Windows 10's telemetry include sending *.doc files if Word crashed?

Why do stocks necessarily drop during a recession?

Is there a standard way to treat events with unknown times (missing time data)?

Why did the villain in the first Men in Black movie care about Earth's Cockroaches?

Dilemma of explaining to interviewer that he is the reason for declining second interview

Is a debit card dangerous in my situation?

Is there a standard way to treat events with unknown times (missing time data)?

Number of hours instead of frequency count in a chi-squared testQuestion on late entry in Survial AnalysisDifference Between Discrete Time Proportional Hazards and Logistic RegressionUsing survival analysis with multiple eventsMissing survival probability estimates and times using R using an Anderson-Gill model for recurrent eventsEvaluation of survival analysis - time to event calculations and cutoffsHow to count person-time in survival analysis with clustered observationsTime-to-event data with low censoringHow to create a cause specific hazard function for a nonparametric Bayesian model for survival analysis?Survival analysis with acute/dynamic predictors

1

$begingroup$

Suppose we are studying some event and the observations are the pairs: time and indicator whether the event has already happened at this time. We have one observation per subject. No events happen before time 0. The event may happen only once for a subject.

Is there a standard way to treat such data? If so then what libraries am I to use in R?

survival

share|cite|improve this question

edited 45 mins ago

Viktor

asked 3 hours ago

ViktorViktor

532516

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  @AdamO that would seem to be the answer (i.e. that the poster has interval censored survival data), and perhaps you should post it as such.
  $endgroup$
  – Weiwen Ng
  3 hours ago
  

  
  
  
  

add a comment | 

1

$begingroup$

Suppose we are studying some event and the observations are the pairs: time and indicator whether the event has already happened at this time. We have one observation per subject. No events happen before time 0. The event may happen only once for a subject.

Is there a standard way to treat such data? If so then what libraries am I to use in R?

survival

share|cite|improve this question

edited 45 mins ago

Viktor

asked 3 hours ago

ViktorViktor

532516

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  @AdamO that would seem to be the answer (i.e. that the poster has interval censored survival data), and perhaps you should post it as such.
  $endgroup$
  – Weiwen Ng
  3 hours ago
  

  
  
  
  

add a comment | 

$begingroup$

Suppose we are studying some event and the observations are the pairs: time and indicator whether the event has already happened at this time. We have one observation per subject. No events happen before time 0. The event may happen only once for a subject.

Is there a standard way to treat such data? If so then what libraries am I to use in R?

survival

share|cite|improve this question

edited 45 mins ago

Viktor

asked 3 hours ago

ViktorViktor

532516

$endgroup$

share|cite|improve this question

edited 45 mins ago

Viktor

asked 3 hours ago

ViktorViktor

532516

share|cite|improve this question

edited 45 mins ago

Viktor

asked 3 hours ago

ViktorViktor

532516

asked 3 hours ago

ViktorViktor

532516

asked 3 hours ago

ViktorViktor

532516

2 Answers
2

active

oldest

votes

2

$begingroup$

Survival analysis will meet your needs. It has the added benefit of managing left-, right-, and interval-censored events. Not everyone has to die on your watch. Not everyone has to be alive at the beginning. And some mysterious unrecorded deaths can occur, which you only discover long after the fact. All such semi problematic data will be used. The result of the analysis is a properly-modeled (Weibull family of distributions) and optimized (MLE) hazard function which then has predictive power.

If you want to implement this yourself, you can follow the excellent Wikipedia page https://en.m.wikipedia.org/wiki/Survival_analysis,
which includes a full loss function if you want to implement the log-likelihood optimization yourself (with $optim$ in R) AND it includes a brief tutorial on R’s $survival$ package, which I have found more difficult to use than the loss functions, but that may say more about me than the $survival$ package.

share|cite|improve this answer

edited 1 hour ago

answered 3 hours ago

Peter LeopoldPeter Leopold

612115

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you for the answer. Could you mention R libraries that can be used for such data?
  $endgroup$
  – Viktor
  2 hours ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  @Viktor so ordered.
  $endgroup$
  – Peter Leopold
  1 hour ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  What has been your difficulty with the survival package?
  $endgroup$
  – AdamO
  1 hour ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  I was never sure how to specify mixture data: left- and right- and interval-censored in the same data set. The documentation isn't at all clear. It seems you can pick one, but not all three. The Surv object uses a "type" attribute which is a string, not an array. I never did find the work-around in the documentation or the package, but I'm sure there has to be one. I can't debug someone else's functional spec, and I can't debug someone else's documentation, but I can debug my own implementation, so the decision to write from scratch was very, very, very easy.
  $endgroup$
  – Peter Leopold
  32 mins ago
  
  
  

  
  
  
  

add a comment | 

2

$begingroup$

When the time variable indicates the event has happened exactly at that time, the dyad of time and event yes/no is called a survival observation. Censoring occurs when a subject is not under observation for a period of time when they are at risk for the event: if the event happened you wouldn't know. For cases like death or non-recurrent events, you cannot have left or interval censored data, because the subject is never at risk for death again.

Censored data can be included in a survival analysis using a parametric or Cox proportional hazards model. In the Cox model, a semiparametric likelihood is used. One inspects only the distribution of people at risk at any event time. This set is called a risk set. If 1 person dies at time 10, everyone who is alive at time 10 comprises the risk set for that time. Censored observations are merely omitted from risk sets.

It is a different thing to say that a subject enters a study or re-enters a study at a point in time and you know that the event has happened exactly once (assuming no reoccurrence of event). This is missing survival data.

You can use typical methods of missing data, like trying to impute the event time. You can also discretize the intervals which leads to a logistic regression model with an offset for the log of time under observation and a 0/1 response for whether the event occurred.

share|cite|improve this answer

answered 1 hour ago

AdamOAdamO

33.7k263140

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Ok, I have the case of missing survival data. The event may happen only once for a subject. The problem is that all the time data is missing. Are there methods other than imputation to treat such data? May be some likelihood approach?
  $endgroup$
  – Viktor
  1 hour ago
  
  
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  @Viktor there is the EM algorithm if you use parametric survival.
  $endgroup$
  – AdamO
  52 mins ago
  

  
  
  
  

add a comment | 

Your Answer

StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "65"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f395075%2fis-there-a-standard-way-to-treat-events-with-unknown-times-missing-time-data%23new-answer', 'question_page');
}
);

Post as a guest

Name

Email

Required, but never shown

2

$begingroup$

Survival analysis will meet your needs. It has the added benefit of managing left-, right-, and interval-censored events. Not everyone has to die on your watch. Not everyone has to be alive at the beginning. And some mysterious unrecorded deaths can occur, which you only discover long after the fact. All such semi problematic data will be used. The result of the analysis is a properly-modeled (Weibull family of distributions) and optimized (MLE) hazard function which then has predictive power.

If you want to implement this yourself, you can follow the excellent Wikipedia page https://en.m.wikipedia.org/wiki/Survival_analysis,
which includes a full loss function if you want to implement the log-likelihood optimization yourself (with $optim$ in R) AND it includes a brief tutorial on R’s $survival$ package, which I have found more difficult to use than the loss functions, but that may say more about me than the $survival$ package.

share|cite|improve this answer

edited 1 hour ago

answered 3 hours ago

Peter LeopoldPeter Leopold

612115

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you for the answer. Could you mention R libraries that can be used for such data?
  $endgroup$
  – Viktor
  2 hours ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  @Viktor so ordered.
  $endgroup$
  – Peter Leopold
  1 hour ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  What has been your difficulty with the survival package?
  $endgroup$
  – AdamO
  1 hour ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  I was never sure how to specify mixture data: left- and right- and interval-censored in the same data set. The documentation isn't at all clear. It seems you can pick one, but not all three. The Surv object uses a "type" attribute which is a string, not an array. I never did find the work-around in the documentation or the package, but I'm sure there has to be one. I can't debug someone else's functional spec, and I can't debug someone else's documentation, but I can debug my own implementation, so the decision to write from scratch was very, very, very easy.
  $endgroup$
  – Peter Leopold
  32 mins ago
  
  
  

  
  
  
  

add a comment | 

2

$begingroup$

Survival analysis will meet your needs. It has the added benefit of managing left-, right-, and interval-censored events. Not everyone has to die on your watch. Not everyone has to be alive at the beginning. And some mysterious unrecorded deaths can occur, which you only discover long after the fact. All such semi problematic data will be used. The result of the analysis is a properly-modeled (Weibull family of distributions) and optimized (MLE) hazard function which then has predictive power.

If you want to implement this yourself, you can follow the excellent Wikipedia page https://en.m.wikipedia.org/wiki/Survival_analysis,
which includes a full loss function if you want to implement the log-likelihood optimization yourself (with $optim$ in R) AND it includes a brief tutorial on R’s $survival$ package, which I have found more difficult to use than the loss functions, but that may say more about me than the $survival$ package.

share|cite|improve this answer

edited 1 hour ago

answered 3 hours ago

Peter LeopoldPeter Leopold

612115

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you for the answer. Could you mention R libraries that can be used for such data?
  $endgroup$
  – Viktor
  2 hours ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  @Viktor so ordered.
  $endgroup$
  – Peter Leopold
  1 hour ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  What has been your difficulty with the survival package?
  $endgroup$
  – AdamO
  1 hour ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  I was never sure how to specify mixture data: left- and right- and interval-censored in the same data set. The documentation isn't at all clear. It seems you can pick one, but not all three. The Surv object uses a "type" attribute which is a string, not an array. I never did find the work-around in the documentation or the package, but I'm sure there has to be one. I can't debug someone else's functional spec, and I can't debug someone else's documentation, but I can debug my own implementation, so the decision to write from scratch was very, very, very easy.
  $endgroup$
  – Peter Leopold
  32 mins ago
  
  
  

  
  
  
  

add a comment | 

$begingroup$

Survival analysis will meet your needs. It has the added benefit of managing left-, right-, and interval-censored events. Not everyone has to die on your watch. Not everyone has to be alive at the beginning. And some mysterious unrecorded deaths can occur, which you only discover long after the fact. All such semi problematic data will be used. The result of the analysis is a properly-modeled (Weibull family of distributions) and optimized (MLE) hazard function which then has predictive power.

If you want to implement this yourself, you can follow the excellent Wikipedia page https://en.m.wikipedia.org/wiki/Survival_analysis,
which includes a full loss function if you want to implement the log-likelihood optimization yourself (with $optim$ in R) AND it includes a brief tutorial on R’s $survival$ package, which I have found more difficult to use than the loss functions, but that may say more about me than the $survival$ package.

share|cite|improve this answer

edited 1 hour ago

answered 3 hours ago

Peter LeopoldPeter Leopold

612115

$endgroup$

share|cite|improve this answer

edited 1 hour ago

answered 3 hours ago

Peter LeopoldPeter Leopold

612115

answered 3 hours ago

Peter LeopoldPeter Leopold

612115

answered 3 hours ago

Peter LeopoldPeter Leopold

612115

2

$begingroup$

When the time variable indicates the event has happened exactly at that time, the dyad of time and event yes/no is called a survival observation. Censoring occurs when a subject is not under observation for a period of time when they are at risk for the event: if the event happened you wouldn't know. For cases like death or non-recurrent events, you cannot have left or interval censored data, because the subject is never at risk for death again.

Censored data can be included in a survival analysis using a parametric or Cox proportional hazards model. In the Cox model, a semiparametric likelihood is used. One inspects only the distribution of people at risk at any event time. This set is called a risk set. If 1 person dies at time 10, everyone who is alive at time 10 comprises the risk set for that time. Censored observations are merely omitted from risk sets.

It is a different thing to say that a subject enters a study or re-enters a study at a point in time and you know that the event has happened exactly once (assuming no reoccurrence of event). This is missing survival data.

You can use typical methods of missing data, like trying to impute the event time. You can also discretize the intervals which leads to a logistic regression model with an offset for the log of time under observation and a 0/1 response for whether the event occurred.

share|cite|improve this answer

answered 1 hour ago

AdamOAdamO

33.7k263140

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Ok, I have the case of missing survival data. The event may happen only once for a subject. The problem is that all the time data is missing. Are there methods other than imputation to treat such data? May be some likelihood approach?
  $endgroup$
  – Viktor
  1 hour ago
  
  
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  @Viktor there is the EM algorithm if you use parametric survival.
  $endgroup$
  – AdamO
  52 mins ago
  

  
  
  
  

add a comment | 

2

$begingroup$

When the time variable indicates the event has happened exactly at that time, the dyad of time and event yes/no is called a survival observation. Censoring occurs when a subject is not under observation for a period of time when they are at risk for the event: if the event happened you wouldn't know. For cases like death or non-recurrent events, you cannot have left or interval censored data, because the subject is never at risk for death again.

Censored data can be included in a survival analysis using a parametric or Cox proportional hazards model. In the Cox model, a semiparametric likelihood is used. One inspects only the distribution of people at risk at any event time. This set is called a risk set. If 1 person dies at time 10, everyone who is alive at time 10 comprises the risk set for that time. Censored observations are merely omitted from risk sets.

It is a different thing to say that a subject enters a study or re-enters a study at a point in time and you know that the event has happened exactly once (assuming no reoccurrence of event). This is missing survival data.

You can use typical methods of missing data, like trying to impute the event time. You can also discretize the intervals which leads to a logistic regression model with an offset for the log of time under observation and a 0/1 response for whether the event occurred.

share|cite|improve this answer

answered 1 hour ago

AdamOAdamO

33.7k263140

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Ok, I have the case of missing survival data. The event may happen only once for a subject. The problem is that all the time data is missing. Are there methods other than imputation to treat such data? May be some likelihood approach?
  $endgroup$
  – Viktor
  1 hour ago
  
  
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  @Viktor there is the EM algorithm if you use parametric survival.
  $endgroup$
  – AdamO
  52 mins ago
  

  
  
  
  

add a comment | 

$begingroup$

When the time variable indicates the event has happened exactly at that time, the dyad of time and event yes/no is called a survival observation. Censoring occurs when a subject is not under observation for a period of time when they are at risk for the event: if the event happened you wouldn't know. For cases like death or non-recurrent events, you cannot have left or interval censored data, because the subject is never at risk for death again.

Censored data can be included in a survival analysis using a parametric or Cox proportional hazards model. In the Cox model, a semiparametric likelihood is used. One inspects only the distribution of people at risk at any event time. This set is called a risk set. If 1 person dies at time 10, everyone who is alive at time 10 comprises the risk set for that time. Censored observations are merely omitted from risk sets.

It is a different thing to say that a subject enters a study or re-enters a study at a point in time and you know that the event has happened exactly once (assuming no reoccurrence of event). This is missing survival data.

You can use typical methods of missing data, like trying to impute the event time. You can also discretize the intervals which leads to a logistic regression model with an offset for the log of time under observation and a 0/1 response for whether the event occurred.

share|cite|improve this answer

answered 1 hour ago

AdamOAdamO

33.7k263140

$endgroup$

share|cite|improve this answer

answered 1 hour ago

AdamOAdamO

33.7k263140

answered 1 hour ago

AdamOAdamO

33.7k263140

answered 1 hour ago

AdamOAdamO

33.7k263140