I’m working on a custom CTGAN-like model for tabular data, where I implemented my own _collate_fn and run_step() logic to control sampling, noise injection, and discriminator inputs. However, I consistently run into shape mismatch errors between the real and fake data batches during training — particularly in the discriminator step. What I know so far: I already apply drop_last=True in my DataLoader. I try to slice all tensors in _collate_fn to a shared min_len, including disc_in_known, disc_in_unknown, disc_in_fakez, disc_in_c, disc_in_perm. In run_step(), I slice real/fake tensors before loss calculation. Still, I get errors like: ValueError: shape mismatch: real_mean=torch.Size([6823]), fake_mean=torch.Size([6814]) I use a custom collate_fn() because i need to sample multiple times per batch for the discriminaton. Inject Gaussian noise and slice all inputs to the same minimum lenght to avoid mismatches. This function prepares both the discriminator and generator inputs before passing them into the training loop. def _collate_fn(self, batch: List[Tuple[Tensor, ...]]) -> Tuple[Tensor, ...]: batch_size = len(batch) mean = torch.zeros(batch_size, self._embedding_dim) std = mean + 1 disc_in_known, disc_in_unknown, disc_in_fakez = [], [], [] disc_in_c, disc_in_perm = [], [] for _ in range(self._discriminator_step): fakez = torch.normal(mean=mean, std=std) c1, m1, col, opt = self._sampler.sample_condvec(batch_size) known, unknown = self._sampler.sample_data(batch_size, col, opt) perm = torch.randperm(batch_size) disc_in_known.append(known) disc_in_unknown.append(unknown) disc_in_fakez.append(fakez) disc_in_c.append(c1) disc_in_perm.append(perm) # Use min_len to align shapes across all tensors min_len = min(k.shape[0] for k in disc_in_known + disc_in_fakez + disc_in_c + disc_in_perm + disc_in_unknown) disc_in_known = torch.stack([k[:min_len] for k in disc_in_known]) disc_in_unknown = torch.stack([k[:min_len] for k in disc_in_unknown]) disc_in_fakez = torch.stack([k[:min_len] for k in disc_in_fakez]) disc_in_c = torch.stack([k[:min_len] for k in disc_in_c]) disc_in_perm = torch.stack([k[:min_len] for k in disc_in_perm]) # generator input gen_in_fakez = torch.normal(mean=mean, std=std) c1, m1, col, opt = self._sampler.sample_condvec(batch_size) known, unknown = self._sampler.sample_data(batch_size, col, opt) return disc_in_known, disc_in_unknown, disc_in_fakez, disc_in_c, disc_in_perm, known, unknown, gen_in_fakez, c1, m1ü Despite all the slicing and drop_last=True, I still get shape mismatch errors during training. Could there be another source of inconsistency I’m overlooking? Any suggestions on how to ensure real and fake batches always align would be appreciated. Continue reading...
